Getting Started

Latest review: 2022-04-04

Installation

To complete the installation we will need to perform the following steps:

1. install python3 and pip
2. install aprendizations
3. generate SSL certificates
4. configure the firewall (optional)

To use the software we need to:

1. initialize a database
2. go to the demo directory (or an existing course)
3. run aprendizations courses.yaml

Each of these steps is explained below.

Install python3 with sqlite3 support

Minimum supported version is python3.8. The installed versions depend on the operating system default.

sudo pkg install python3 py38-sqlite3   # FreeBSD
sudo apt install python3                # Linux (Ubuntu)
sudo port install python310             # MacOS

Install pip

Install pip from the system package manager:

sudo pkg install py38-pip          # FreeBSD
sudo apt install python3-pip       # Linux (Ubuntu)
sudo port install py310-pip        # MacOS

In the end you should be able to run pip --version and python3 -c "import sqlite3" without errors. In some systems, pip can be named pip3, pip3.8 or pip-3.8, etc.

Packages should not be installed system-wide. Either install them to a python virtual environment or in the user area.

To install in the user area use pip install --user some_package or edit the configuration file ~/.config/pip/pip.conf (FreeBSD, Linux) or Library/Application Support/pip/pip.conf (MacOS) and add the lines

[global]
user = yes

Install aprendizations

pip install git+https://git.xdi.uevora.pt/mjsb/aprendizations.git

Python packages are usually installed in:

• ~/.local/lib/python3.8/site-packages/ in Linux/FreeBSD.
• ~/Library/python/3.10/lib/python/site-packages/ in MacOS.

When aprendizations is installed with pip, all the dependencies are also installed.

At this point, the commands aprendizations and initdb-aprendizations are installed in

~/.local/bin                          # Linux/FreeBSD
~/Library/Python/3.10/bin             # MacOS

and can be run from the terminal:

aprendizations --version
aprendizations --help
initdb-aprendizations --help

SSL Certificates

We need certificates for https. Certificates can be self-signed or validated by a trusted authority.

Self-signed can be used locally for development and testing, but browsers will complain. LetsEncrypt issues trusted and free certificates, but the server must have a registered publicly accessible domain name.

Generating selfsigned certificates

Generate a selfsigned certificate and place it in ~/.local/share/certs.

openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes

LetsEncrypt certificates

Install the certbot from LetsEncrypt:

sudo pkg install py38-certbot       # FreeBSD
sudo apt install certbot            # Ubuntu

To generate or renew the certificates, ports 80 and 443 must be accessible. Any firewall and webserver have to be stopped.

sudo certbot certonly --standalone -d www.example.com   # first time
sudo certbot renew                                      # renew

Certificates are saved under /usr/local/etc/letsencrypt/live/www.example.com/. Copy them to ~/.local/share/certs and change permissions to be readable:

cd ~/.local/share/certs
sudo cp /usr/local/etc/letsencrypt/live/www.example.com/cert.pem .
sudo cp /usr/local/etc/letsencrypt/live/www.example.com/privkey.pem .
chmod 400 cert.pem privkey.pem

Configuration

Database

User data is maintained in a sqlite3 database which has to be created manually using the initdb-aprendizations command. The database file should be located in the same directory as the main YAML configuration file (courses.yaml).

For example, to run the included demo do:

cd demo                               # contains a small example
initdb-aprendizations                 # show or initialize database
initdb-aprendizations --admin         # add admin user
initdb-aprendizations inscricoes.csv  # add students from CSV
initdb-aprendizations --add 1184 "Aladino da Silva"   # add new user
initdb-aprendizations --update 1184 --pw alibaba      # update password
initdb-aprendizations --help          # for available options

The default password is equal to the user name, if left undefined.

Running the demo

The application includes a small example in demo/courses.yaml that can be used for initial testing. Run it with

cd demo
aprendizations courses.yaml

Open the browser at https://127.0.0.1:8443. If everything looks good, check at the correct address https://www.example.com:8443. The option --debug provides more verbose logging and might be useful during testing.

Firewall configuration

Ports 80 and 443 are only usable by root. For security reasons the server runs as an unprivileged user on port 8443 for https. To access the server in the default https port (443), port forwarding must be configured in the firewall.

FreeBSD and pf

Edit /etc/pf.conf:

ext_if="em0"                # change em0 to the correct network interface
rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443

Virtualbox with guest additions uses ext_if="vtnet0".

Edit /etc/rc.conf

pf_enable="YES"
pf_flags=""
pf_rules="/etc/pf.conf"

# optional logging:
pflog_enable="YES"
pflog_flags=""
pflog_logfile="/var/log/pflog"

Reboot or sudo service pf start.

Example configuration files are in the freebsd directory.

Testing the system

Make sure the following steps have been done:

• installed python3 and pip
• installed aprendizations using pip
• initialized database with at least 1 user
• generate and copy certificates to the appropriate place
• (optional) configure the firewall to do port forwarding
• run aprendizations courses.yaml --check

Keeping aprendizations updated

To update aprendizations to the latest version do:

pip install -U git+https://git.xdi.uevora.pt/mjsb/aprendizations.git

Troubleshooting

To help with troubleshooting, use the option --debug when running the server. This will increase logs in the terminal and will present the python exception errors in the browser.

Logging levels can be adjusted in ~/.config/aprendizations/logger.yaml and ~/.config/aprendizations/logger-debug.yaml.

If these files do not yet exist, there are examples in aprendizations/config that can be copied to ~/.config/aprendizations.

UnicodeEncodeError

The server should not generate this error, but when using external scripts to generate questions or to correct, these scripts can print unicode strings to stdout. If the terminal does not support unicode, python will generate an exception.

• FreeBSD fix: edit ~/.login_conf to use UTF-8, for example:

me:\
    :charset=UTF-8:\
    :lang=en_US.UTF-8:

• Debian fix: check locale...

The application runs but questions do not show up

Some operating systems have an option to disable animations to try to avoid motion sickness in some people. Browsers will check this option with the OS and prevent animate.css library to work. Since questions have several animations, these will will not work and nothing is shown on the page.

To fix this issue you need to allow animations in the Operating System:

• On windows 10, go to System Preferences, search for "Show animations in windows" and turn it ON.
• On MacOS or iOS search for reduced motion and switch it OFF (Preferences -> Acessibility -> Display -> Reduce motion).

FAQ

Common database manipulations:

initdb-aprendizations -u 12345 --pw alibaba          # reset student password
initdb-aprendizations -a 007 "James Bond" --pw mi6   # add new student "007"

Some common database queries:

# Which students did at least one topic?
sqlite3 students.db "select distinct student_id from studenttopic"

# How many topics has each student done?
sqlite3 students.db "select student_id, count(topic_id) from studenttopic \
  group by student_id order by count(topic_id) desc"

# Which questions have more wrong answers?
sqlite3 students.db "select count(ref), ref from answers where grade<1.0 \
  group by ref order by count(ref) desc"