README

1. Installation
2. Running a demo
3. Troubleshooting

1. Installation

1.1 Requirements

The webserver is a python application and requires python3.7 and pip to be installed, plus the following additional packages:

• tornado
• mistune
• PyYAML
• Pygments
• SQLAlchemy
• bcrypt

These can be installed for a single user, in a python virtual environment or system wide.

Installing packages for a single user

pip3 install --user tornado mistune pyyaml pygments sqlalchemy bcrypt

Installing packages in a virtual environment (alternative)

python3 -m venv PATH/TO/VENV/perguntations
source PATH/TO/VENV/perguntations/bin/activate     # activate virtualenv

pip3 install tornado mistune pyyaml pygments sqlalchemy bcrypt markdown

Installing packages system wide (alternative)

I personally prefer python packages to be installed for a single user, but if a system wide installation is required, the operating system package manager should be used:

Linux:

apt install python3-tornado python3-mistune python3-yaml python3-pygments...

macOS macports:

port install py37-tornado py37-mistune py37-yaml py37-pygments...

FreeBSD:

pkg install py37-tornado py37-mistune py37-yaml py37-pygments...

1.2 Install and setup perguntations

There is no installer, pip or otherwise. It's still in development and has to be run from sources. Use git to get the sources:

cd WHERE/TO/PUT/THE/SOFTWARE
git clone https://USERNAME@bitbucket.org/USERNAME/perguntations.git

where USERNAME is your account on bitbucket.

The server will run an https server and requires valid certificates. To generate certificates, there are two possibilities: public server with static IP address or a private server.

The thirdparty javascript libraries are not included in the repository. They should be downloaded and installed in the directory perguntations/static/lib.

The following libraries are currently necessary (as of 24/1/2019):

DataTables
MDB-Free_4
MathJax-2.7.5
bootstrap-4.2.1-dist
codemirror-5.42.2
fontawesome-free-5.6.3-web
jquery-3.3.1.min.js
popper.min.js

Generating certificates for a public server (FreeBSD)

sudo pkg install py27-certbot       # FreeBSD

Shutdown the firewall and any server running. Then run the script to generate the certificate:

sudo service pf stop                            # disable pf firewall (FreeBSD)
sudo certbot certonly --standalone -d www.example.com
sudo service pf start                           # enable pf firewall

Certificates are saved under /usr/local/etc/letsencrypt/live/www.example.com/. Copy them to aprendizations/certs and change permissions to be readable:

sudo cp /usr/local/etc/letsencrypt/live/www.example.com/cert.pem .
sudo cp /usr/local/etc/letsencrypt/live/www.example.com/privkey.pem .
chmod 400 cert.pem privkey.pem

Renews can be done as follows:

sudo service pf stop   # shutdown firewall
sudo certbot renew
sudo service pf start  # start firewall

Copy certificates cert.crt and cert.key to the perguntations/certs/ directory.

Generating self-signed certificates

Self-signed certificates are not certified by a recognised authority and browsers will complain that the certificate is not trusted.

cd perguntations/certs/
openssl req -x509 -newkey rsa:4096 -keyout cert.key -out cert.crt -days 365 -nodes

Running a demo

The directory demo includes a demo test that can be used as a template for your own tests and questions.

To run the demonstration test:

cd WHERE/YOU/INSTALLED/perguntations

# create and initialize database using one of the following methods:
./initdb.py students.csv    # initialize from a CSV file
./initdb.py --admin         # only adds the administrator account
./initdb.py --add 123 "Asterix Gaules"  # add one student

# a database file "students.db" is created
mv students.db demo/

# create directory to save the finished tests
mkdir demo/ans

# edit test configuration and check if everything looks right
vi demo/test-tutorial.yaml

We are now ready to run the server:

./serve.py --help                     # get help
./serve.py demo/test-tutorial.yaml    # run demo test

By default the server listens on port 8443 and on all IPs of all network interfaces. Open the browser at http://127.0.0.1:8443/ and login as user number 0 (administrator) and choose any password. Then

1. Authorize students by clicking the checkboxes.
2. Open a different browser at http://127.0.0.1:8443/ and login as one of the authorized students. Answer the questions and submit.

The server can be stoped from the terminal with ^C.

Running on lower ports 80 or 443

Ports 80 and 443 are reserved for the root user and and this software should NOT be run as root. Instead, the firewall should be configured to forward tcp traffic from port 443 to 8443 where the server is listening. The details depend on the operating system.

debian:

iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-ports 8080
FIXME: also for port 443...

Explanation:

• -t nat is the table consulted when a packet creates a new connection.
• -I PREROUTING inserts rules at the head of the chain.
• -p tcp selected protocol.
• -s 0/0 source network address/mask.
• -i eth0 interface via which the packet was received.
• --dport 80 destination port.
• -j REDIRECT what to do when packet matches the rule.
• --to-ports 8080 where to redirect the packets.

FreeBSD and pf

Edit /etc/pf.conf:

ext_if="em0"
rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443

or ext_if="vtnet0" for guest additions under virtual box.

Start firewall with sudo service pf start.

Optionally, to activate pf on boot, edit rc.conf:

pf_enable="YES"
pf_flags=""
pf_rules="/etc/pf.conf"

# optional logging:
pflog_enable="YES"
pflog_flags=""
pflog_logfile="/var/log/pflog"

Troubleshooting

• The server tries to run python3 so this command must be accessible from user accounts. Currently, the minimum supported python version is 3.6.

• If you are getting any UnicodeEncodeError type of errors that's because the terminal is not supporting UTF-8. This error may occur when a unicode character is printed to the screen by the server or, when running question generator or correction scripts, a message is piped between the server and the scripts that includes unicode characters. Try running locale on the terminal and see if there are any error messages. Solutions:

  • debian: fix it with sudo dpkg-reconfigure locales, select your UTF-8 locales and try again.

  • FreeBSD: edit ~/.login_conf to use UTF-8, for example:

    me:\
            :charset=UTF-8:\
            :lang=en_US.UTF-8:
    

Contribute

• Writing questions in yaml format
• Testing and reporting bugs
• Code review
• New features and ideas

Contacts

• Miguel Barão mjsb@uevora.pt