# Getting Started

## Requirements

Before installing the server, we will need to install python3.7 with some additional packages, and npm (Node package management).

### Install python3.7 with sqlite3 support and npm

This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources.

- Installing from the system package manager:

```sh
sudo port install python37 npm5              # MacOS

sudo pkg install python37 py37-sqlite3 npm   # FreeBSD

sudo apt install python3.7 npm               # Linux

```

- Installing from source:

Download from [http://www.python.org]() and

```sh
unxz Python-3.7.tar.xz

tar xvf Python-3.7.tar
cd Python-3.7

./configure --prefix=$HOME/.local/bin
make && make install

```

This will install python locally under `~/.local/bin`. Make sure to add it to your `PATH` (edit `~/.profile` in MacOS or FreeBSD).

### Install pip

If the `pip` command is not yet installed, run

```sh

python3.7 -m ensurepip --user

```

This will install pip in your user account under `~/.local/bin`.
In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.7` or `pip-3.7`).

If you want to always install python modules on the user account (recommended), edit the pip configuration file `~/.config/pip/pip.conf` (FreeBSD, Linux) or `Library/Application Support/pip/pip.conf` (MacOS) and add the lines

```ini
[global]
user = yes
```

### Install python packages and javascript libraries:

Replace USER by your bitbucket username:

```sh

cd path/to/some/directory

git clone https://USER@bitbucket.org/mjsb/aprendizations.git

cd aprendizations

pip install .           # install aprendizations and dependencies

npm install             # install javascript libraries

```

Python packages are usually installed in:

- `~/.local/lib/python3.7/site-packages/` in Linux/FreeBSD.

- `~/Library/python/3.7/lib/python/site-packages/` in MacOS.

Javascript libraries are installed in `aprendizations/node_modules` directory.

This libraries are linked from the `aprendizations/aprendizations/static` directory.

At this point aprendizations can be run anywhere with the command `aprendizations`.

## Configuration

### Database

The user data is maintained in a sqlite3 database file. We first need to create the database:

```sh

cd demo

initdb-aprendizations                 # show or initialize database
initdb-aprendizations --admin         # add admin user
initdb-aprendizations inscricoes.csv  # add students from CSV

initdb-aprendizations --add 1184 "Aladino da Silva"   # add user

initdb-aprendizations --update 1184 --pw alibaba      # update password

initdb-aprendizations --help          # for the available options

```

The default password is equal to the user name used to login.

### SSL Certificates

We need certificates for https. Certificates can be self-signed or validated by a trusted authority.

Self-signed can be used locally for development and testing, but browsers will complain.

LetsEncrypt issues trusted and free certificates, but the server must have a registered publicly accessible domain name.

#### Selfsigned

Generate a selfsigned certificate and place it in `aprendizations/certs`.

```sh

openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes
```

#### LetsEncrypt

```sh

sudo pkg install py27-certbot       # FreeBSD

```

Shutdown the firewall and any web server that might be running. Then run the script to generate the certificate:

```sh

sudo service pf stop                            # disable pf firewall (FreeBSD)

sudo certbot certonly --standalone -d www.example.com
sudo service pf start                           # enable pf firewall

```

Certificates are saved under `/usr/local/etc/letsencrypt/live/www.example.com/`. Copy them to `aprendizations/certs` and change permissions to be readable:

```sh

sudo cp /usr/local/etc/letsencrypt/live/www.example.com/cert.pem .

sudo cp /usr/local/etc/letsencrypt/live/www.example.com/privkey.pem .

chmod 400 cert.pem privkey.pem

```

Renews can be done as follows:

```sh

sudo service pf stop   # shutdown firewall
sudo certbot renew
sudo service pf start  # start firewall
```

and then copy the `cert.pem` and `privkey.pem` files to `aprendizations/certs` directory. Change permissions and ownership as appropriate.

### Testing

The application includes a small example in `demo/demo.yaml`. Run it with

```sh
cd demo
aprendizations demo.yaml
```

and open a browser at [https://127.0.0.1:8443](). If it everything looks good, check at the correct address `https://www.example.com` (requires port forward in the firewall). The option `--debug` provides more verbose logging and might be useful during testing.


### Firewall configuration

Ports 80 and 443 are only usable by root. For security reasons it is better to run the server as an unprivileged user on higher ports like 8080 for http and 8443 for https. For this, we can configure port forwarding in the firewall to redirect incoming tcp traffic from 80 to 8080 and 443 to 8443.

#### FreeBSD and pf

Edit `/etc/pf.conf`:

```sh

ext_if="em0"                # this should be the correct network interface

rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080

rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443

```

Under virtualbox with guest additions use `ext_if="vtnet0"`.

Edit `/etc/rc.conf`

```sh
pf_enable="YES"

pf_flags=""
pf_rules="/etc/pf.conf"

# optional logging:
pflog_enable="YES"
pflog_flags=""
pflog_logfile="/var/log/pflog"
```

Reboot or `sudo service pf start`.

## Troubleshooting

To help with troubleshooting, use the option `--debug` when running the server. This will increase logs in the terminal and will present the python exception errors in the browser.

Logging levels can be adjusted in `~/.config/aprendizations/logger.yaml` and `~/.config/aprendizations/logger-debug.yaml`.

#### UnicodeEncodeError

The server should not generate this error, but when using external scripts to generate questions or to correct, these scripts can print unicode strings to stdout. If the terminal does not support unicode, python will generate this exception.

- FreeBSD fix: edit `~/.login_conf` to use UTF-8, for example:

```sh
me:\
    :charset=UTF-8:\

    :lang=en_US.UTF-8:

```

- Debian fix: check `locale`...

## FAQ

- Which students did at least one topic?

```sh
sqlite3 students.db "select distinct student_id from studenttopic"

```

- How many topics has each student done?

```sh

sqlite3 students.db "select student_id, count(topic_id) from studenttopic group by student_id order by count(topic_id) desc"
```

- Which questions have more wrong answers?

```sh

sqlite3 students.db "select count(ref), ref from answers where grade<1.0 group by ref order by count(ref) desc"
```