# Get Started


## Requirements

We will need to install python3.6 with sqlite3 support.
This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources.

- Installing from the system package manager:
    - OSX: `port install python36`
    - FreeBSD: `pkg install python36 py36-sqlite3`
    - Linux: `apt-get install ???` (not available yet?)
- Installing from source:
    - Download from [http://www.python.org]()
    - `unxz Python-3.6.tar.xz`
    - `tar xvf Python-3.6.tar`
    - `cd Python-3.6`
    - `./configure --prefix=$HOME/.local/bin`
    - `make && make install`

This will install python locally under `~/.local/bin`. Make sure to add it to your `PATH` (edit `~/.profile` in OSX or FreeBSD).

Next install pip (if not yet installed):

    python3.6 -m ensurepip --user

This will install pip in your account under `~/.local/bin`.
In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.6` or `pip-3.6`).

Install additional python packages locally on the user area:

    pip3 install --user \
        tornado \
        sqlalchemy \
        pyyaml \
        pygments \
        markdown \
        bcrypt \
        networkx

These are usually installed under

- OSX: `~/Library/python/3.6/lib/python/site-packages/`
- Linux/FreeBSD: `~/.local/lib/python3.6/site-packages/`

Note: If you want to always install python modules on the user account, edit the pip configuration file `~/.config/pip/pip.conf` (FreeBSD, Linux) or `Library/Application Support/pip/pip.conf` (OSX) and add the lines

    [global]
    user = yes

## Installation

Replace USER by your bitbucket username:

    cd path/to/some/directory
    git clone https://USER@bitbucket.org/mjsb/aprendizations.git

A directory `aprendizations` will be created with the software:

    cd aprendizations

## Configuration

### Database

First we need to create a database:

    ./initdb.py                 # initialize user `0` and empty password
    ./initdb.py --pw alibaba    # initialize user `0` and given password
    ./initdb.py --help          # for the available options

### SSL Certificates

We need certificates for https. Certificates can be self-signed or certificates validated by a trusted authority.
Self-signed can be used for development, but browsers will complain.
LetsEncrypt issues trusted and free certificates, but the served must have a fixed IP and a domain name (not dynamic).

#### Selfsigned

Generate selfsigned certificates using

    openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -nodes

and place them in `aprendizations/certs`.

#### LetsEncrypt

    sudo pkg install py27-certbot       # FreeBSD

Shutdown any server running and the firewall, and then run the script to generate the certificate:

    sudo pfctl -d                                   # disable pf firewall
    sudo certbot certonly --standalone -d bit.xdi.uevora.pt
    sudo pfctl -e; sudo pfctl -f /etc/pf.conf       # enable pf firewall

Certificates are saved under `/usr/local/etc/letsencrypt/live/bit.xdi.uevora.pt/` which is readable only by root.

Copy them to `aprendizations/certs` with names `cert.pem` and `key.pem`. And change permissions to be readble (FIXME how to do it securily?)

### Testing

Run a demonstration:

    ./serve.py demo/demo.yaml

and open a browser at `https://127.0.0.1:8443`.

Logging level can be adjusted in `config/logger.yaml`.

### Firewall configuration

Ports 80 and 443 are only usable by root. For security reasons it is better to run the server as a regular user on higher ports like 8080 for http and 8443 for https. In this case, we should configure port forwarding in the firewall to redirect incoming tcp traffic from 80 to 8080 and 443 to 8443.

#### FreeBSD and pf

Edit `/etc/pf.conf`:

    ext_if="em0"
    rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
    rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443

or `ext_if="vtnet0"` for guest additions under virtual box.

Edit `rc.conf`

    pf_enable="YES"
    pf_flags=""
    pf_rules="/etc/pf.conf"

    # optional logging:
    pflog_enable="YES"
    pflog_flags=""
    pflog_logfile="/var/log/pflog"

Reboot or `sudo service pf start`.

## Troubleshooting

#### UnicodeEncodeError

The server should not generate this error, but when using external scripts to generate questions or to correct, these scripts can print unicode strings to stdout. If the terminal does not support unicode, python will generate this exception.

- FreeBSD fix: edit `~/.login_conf` to use UTF-8, for example:

    me:\
            :charset=UTF-8:\
            :lang=en_US.UTF-8:

- Debian fix: check `locale`...

## Useful sqlite3 queries

Which students did at least one topic?

    sqlite3 students.db "select distinct student_id from studenttopic"


How many topics had each student done?

    sqlite3 students.db "select student_id, count(topic_id) from studenttopic group by student_id order by count(topic_id) desc"


Which questions have more wrong answers?

    sqlite3 students.db "select count(ref), ref from answers where grade<1.0 group by ref order by count(ref) desc"