User Manual

1. Requirements
2. Installation
3. Setup
4. Running a demo
5. Running on lower ports
6. Troubleshooting

Requirements

The webserver is a python application that requires >=python3.8 and the package installer for python pip. The node package management npm is also necessary in order to install the javascript libraries.

sudo apt install python3 python3-pip npm                     # Ubuntu
sudo pkg install python38 py38-sqlite3 py38-pip npm          # FreeBSD
sudo port install python38 py38-pip py38-setuptools npm6     # MacOS (macports)

To make the pip install packages to a local directory, the file pip.conf should be configured as follows:

[global]
user=yes

This file is usually in ~/.config/pip/ in Linux and FreeBSD. In MacOS it's in ~/Library/Application Support/pip/. You may have to create it, if it doesn't exist yet.

Installation

Download and install:

git clone https://git.xdi.uevora.pt/mjsb/perguntations.git
cd perguntations
npm install
pip install .

The command npm installs the javascript libraries and then pip installs the python webserver. This will also install any required dependencies.

Atention! pip install . must be run after npm install, otherwise the javascript libraries will not be found during the install.

Setup

The server will run a https server and requires valid certificates. There are two possibilities to generate the certificates:

• public server with static IP address and registered domain name;
• private server on a local network isolated from the internet.

Certificates must be saved in the $XDG_DATA_HOME path if defined in the environment, otherwise in ~/.local/share/certs (I will assume the latter).

mkdir -p ~/.local/share/certs

Self-signed certificates are not certified by a recognised authority and browsers will complain that the certificate is not trusted. That's ok.

cd ~/.local/share/certs
openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes

To have a true certificate, like the ones issued by LetsEncrypt, the host must have a registered domain name. See instructions for FreeBSD at the end of this page.

Running a demo

The directory demo in the repository includes a demo test that can be used as a template for your own tests and questions. To run the demonstration test you need to initialize the database using one of the following methods:

cd perguntations/demo
initdb students.csv                 # initialize from a CSV file
initdb --admin                      # only adds the administrator account
initdb --add 123 "Asterix Gaules"   # add one student

This will create or update a students.db file that contains a sqlite3 database. The database stores user passwords and grades, but not the actual tests.

A test is specified in a single yaml file. The file demo.yaml is a test specification that you can play with.

The answered tests submitted by the students are stored in JSON files in a directory defined in demo.yaml under the option answers_dir: ans. We also have to create this directory manually:

mkdir ans                           # directory where the tests will be saved

Start the server and run the demo.yaml test:

perguntations demo.yaml             # run demo test

Several options are available, run perguntations --help for a list.

The server listens on port 8443 of all IPs of all network interfaces. Open the browser at http://127.0.0.1:8443/ and login as user number 0 (administrator) and choose any password you want. The password is defined on first login.

After logging in, you will be redirected to the administration page that shows all the students and their current state.

1. Authorize students by clicking the checkboxes.
2. Open a different browser (or exit administrator) at http://127.0.0.1:8443/ and login as one of the authorized students. Answer the questions and submit. You should get a grade at the end.

The server can be stoped from the terminal with ^C (Control+C).

Running on lower ports

Ports 80 and 443 are reserved for the root user and and this software should NOT be run as root. Instead, tcp traffic can be forwarded from port 443 to 8443 where the server is listening. The details depend on the operating system/firewall.

debian (FIXME: Untested)

iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-ports 8443

Explanation:

• -t nat is the table consulted when a packet creates a new connection.
• -I PREROUTING inserts rules at the head of the chain.
• -p tcp selected protocol.
• -s 0/0 source network address/mask.
• -i eth0 interface via which the packet was received.
• --dport 443 destination port.
• -j REDIRECT what to do when packet matches the rule.
• --to-ports 8443 where to redirect the packets.

FreeBSD and pf

Edit /etc/pf.conf:

ext_if="wlan1"
rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443

The wlan1 should be the name of the network interface. Use ext_if="vtnet0" for guest additions under virtual box.

Start firewall with sudo service pf start.

Optionally, to activate pf on boot, edit rc.conf:

pf_enable="YES"
pf_flags=""
pf_rules="/etc/pf.conf"

# optional logging:
pflog_enable="YES"
pflog_flags=""
pflog_logfile="/var/log/pflog"

Generating certificates with LetsEncript (FreeBSD)

Generating certificates for a public server (FreeBSD) requires a registered domain with fixed IP.

sudo pkg install py38-certbot                   # FreeBSD 13
sudo service pf stop                            # disable pf firewall (FreeBSD)
sudo certbot certonly --standalone -d www.example.com
sudo service pf start                           # enable pf firewall

Certificates are saved in /usr/local/etc/letsencrypt/live/www.example.com/. Copy them to the ~/.local/share/certs directory and change permissions:

chmod 400 cert.pem privkey.pem

Certificate renewals can be done as follows:

sudo service pf stop        # shutdown firewall
sudo certbot renew
sudo service pf start       # start firewall

Again, copy certificate files privkey.pem and cert.pem to ~/.local/share/certs.

Upgrading

From time to time there can be updates to perguntations, python packages and javascript libraries. Python packages can be upgraded independently of the rest using pip:

pip list --outdated --user  # lists upgradable packages
pip install -U something    # upgrade something

Attention: don't upgrade blindly. Some upgrades can be incompatible with perguntations and break things! All upgrades require testing of all the funcionalities.

To upgrade perguntations and javascript libraries do:

cd perguntations
git pull                    # get latest version of perguntations
npm update                  # get latest versions of javascript libraries
pip install -U .            # upgrade perguntations

Troubleshooting

• The server tries to run python3 so this command must be accessible from user accounts. Currently, the minimum supported python version is 3.8.

• If you are getting any UnicodeEncodeError type of errors that's because the terminal is not supporting UTF-8. This error may occur when a unicode character is printed to the screen by the server or, when running question generator or correction scripts, a message is piped between the server and the scripts that includes unicode characters. Try running locale on the terminal and see if there are any error messages. Solutions:

  • debian: sudo dpkg-reconfigure locales and select your UTF-8 locales.
  • FreeBSD: edit ~/.login_conf to use UTF-8, for example:

  me:\
          :charset=UTF-8:\
          :lang=en_US.UTF-8:
  

Please contribute

• Writing questions in yaml format
• Testing and reporting bugs
• Code review
• New features and ideas

Contacts

• Miguel Barão mjsb@uevora.pt