Merge branch 'dev' of https://git.xdi.uevora.pt/mjsb/perguntations into dev

Miguel Barão
2 parents 981734c0 0b27f204
Showing 2 changed files with 84 additions and 77 deletions Show diff stats
README.md
package.json
@@ -11,26 +11,27 @@
  
 ## Requirements
  
-The webserver is a python application that requires `>=python3.7` and the package installer for python `pip`. The node package management `npm` is also necessary in order to install the javascript libraries.
+The webserver is a python application that requires `>=python3.8` and the
+package installer for python `pip`. The node package management `npm` is also
+necessary in order to install the javascript libraries.
  
 ```sh
 sudo apt install python3 python3-pip npm                     # Ubuntu
-sudo pkg install python37 py37-sqlite3 py37-pip npm          # FreeBSD
-sudo port install python37 py37-pip py37-setuptools npm6     # MacOS (macports)
+sudo pkg install python38 py38-sqlite3 py38-pip npm          # FreeBSD
+sudo port install python38 py38-pip py38-setuptools npm6     # MacOS (macports)
 ```
  
-To make the `pip` install packages to a local directory, the file `pip.conf` should be configured as follows:
+To make the `pip` install packages to a local directory, the file `pip.conf`
+should be configured as follows:
  
 ```ini
 [global]
 user=yes
-
-[list]
-format=columns
 ```
  
 This file is usually in `~/.config/pip/` in Linux and FreeBSD. In MacOS it's in
-`~/Library/Application Support/pip/`. You may have to create it, if it doesn't exist yet.
+`~/Library/Application Support/pip/`. You may have to create it, if it doesn't
+exist yet.
  
 ---
  
@@ -42,19 +43,21 @@ Download and install:
 git clone https://git.xdi.uevora.pt/mjsb/perguntations.git
 cd perguntations
 npm install
-pip3 install .
+pip install .
 ```
  
-The command `npm` installs the javascript libraries and then `pip3` installs the python webserver. This will also install any required dependencies.
+The command `npm` installs the javascript libraries and then `pip` installs the
+python webserver. This will also install any required dependencies.
  
-**Atention!** `pip3 install .` must run **after** `npm install`, otherwise the javascript libraries will not be found during the install.
+**Atention!** `pip install .` must be run **after** `npm install`, otherwise
+the javascript libraries will not be found during the install.
  
 ---
  
 ## Setup
  
-The server will run a `https` server and requires valid certificates.
-There are two possibilities to generate the certificates:
+The server will run a `https` server and requires valid certificates.  There
+are two possibilities to generate the certificates:
  
 - public server with static IP address and registered domain name;
 - private server on a local network isolated from the internet.
@@ -74,38 +77,35 @@ cd ~/.local/share/certs
 openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes
 ```
  
-To have a true certificate, like the ones issued by LetsEncrypt, the host
-must have a registered domain name.
-See instructions for FreeBSD at the end of this page.
+To have a true certificate, like the ones issued by LetsEncrypt, the host must
+have a registered domain name.  See instructions for FreeBSD at the end of this
+page.
  
 ---
  
 ## Running a demo
  
-The directory `demo` in the repository includes a demo test that can be used
-as a template for your own tests and questions.
-
-To run the demonstration test you need to initialize the database using one of
-the following methods:
+The directory `demo` in the repository includes a demo test that can be used as
+a template for your own tests and questions.  To run the demonstration test you
+need to initialize the database using one of the following methods:
  
 ```sh
 cd perguntations/demo
-
 initdb students.csv                 # initialize from a CSV file
 initdb --admin                      # only adds the administrator account
 initdb --add 123 "Asterix Gaules"   # add one student
 ```
  
 This will create or update a `students.db` file that contains a sqlite3
-database.
-The database stores user passwords and grades, but not the actual tests.
+database.  The database stores user passwords and grades, but not the actual
+tests.
  
-A test is specified in a single `yaml` file.
-The demo already includes the `demo.yaml` that you can play with.
+A test is specified in a single `yaml` file.  The file `demo.yaml` is a test
+specification that you can play with.
  
-The complete tests submitted by the students are stored in JSON files in the
-directory defined in `demo.yaml` under the option `answers_dir: ans`.
-We also have to create this directory manually:
+The answered tests submitted by the students are stored in JSON files in a
+directory defined in `demo.yaml` under the option `answers_dir: ans`.  We also
+have to create this directory manually:
  
 ```sh
 mkdir ans                           # directory where the tests will be saved
@@ -114,38 +114,36 @@ mkdir ans                           # directory where the tests will be saved
 Start the server and run the `demo.yaml` test:
  
 ```sh
-perguntations demo.yaml         # run demo test
+perguntations demo.yaml             # run demo test
 ```
  
 Several options are available, run `perguntations --help` for a list.
  
-The server listens on port 8443 of all IPs of all network interfaces.
-Open the browser at `http://127.0.0.1:8443/` and login as user number `0`
+The server listens on port 8443 of all IPs of all network interfaces.  Open the
+browser at `http://127.0.0.1:8443/` and login as user number `0`
 (administrator) and choose any password you want. The password is defined on
 first login.
+
 After logging in, you will be redirected to the administration page that shows
 all the students and their current state.
  
 1. Authorize students by clicking the checkboxes.
 2. Open a different browser (or exit administrator) at `http://127.0.0.1:8443/`
-and login as one of the authorized students.
-Answer the questions and submit. You should get a grade at the end.
+   and login as one of the authorized students.  Answer the questions and
+   submit. You should get a grade at the end.
  
-The server can be stoped from the terminal with `^C`.
+The server can be stoped from the terminal with `^C` (Control+C).
  
 ---
  
 ## Running on lower ports
  
 Ports 80 and 443 are reserved for the root user and and this software **should
-NOT be run as root**.
-Instead, tcp traffic can be forwarded from port 443 to 8443 where the server is
-listening.
-The details depend on the operating system/firewall.
-
-### debian
+NOT be run as root**.  Instead, tcp traffic can be forwarded from port 443 to
+8443 where the server is listening.  The details depend on the operating
+system/firewall.
  
-FIXME: Untested
+### debian (FIXME: Untested)
  
 ```.sh
 iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-ports 8443
@@ -166,25 +164,29 @@ Explanation:
  
 Edit `/etc/pf.conf`:
  
-    ext_if="wlan1"
-    rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
-    rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
+```conf
+ext_if="wlan1"
+rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
+rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
+```
  
-The `wlan1` should be the name of the network interface.
-Use `ext_if="vtnet0"` for guest additions under virtual box.
+The `wlan1` should be the name of the network interface.  Use `ext_if="vtnet0"`
+for guest additions under virtual box.
  
 Start firewall with `sudo service pf start`.
  
 Optionally, to activate pf on boot, edit `rc.conf`:
  
-    pf_enable="YES"
-    pf_flags=""
-    pf_rules="/etc/pf.conf"
+```conf
+pf_enable="YES"
+pf_flags=""
+pf_rules="/etc/pf.conf"
  
-    # optional logging:
-    pflog_enable="YES"
-    pflog_flags=""
-    pflog_logfile="/var/log/pflog"
+# optional logging:
+pflog_enable="YES"
+pflog_flags=""
+pflog_logfile="/var/log/pflog"
+```
  
 ## Generating certificates with LetsEncript (FreeBSD)
  
@@ -192,7 +194,7 @@ Generating certificates for a public server (FreeBSD) requires a registered
 domain with fixed IP.
  
 ```sh
-sudo pkg install py27-certbot                   # FreeBSD
+sudo pkg install py38-certbot                   # FreeBSD 13
 sudo service pf stop                            # disable pf firewall (FreeBSD)
 sudo certbot certonly --standalone -d www.example.com
 sudo service pf start                           # enable pf firewall
@@ -208,47 +210,52 @@ chmod 400 cert.pem privkey.pem
 Certificate renewals can be done as follows:
  
 ```sh
-sudo service pf stop   # shutdown firewall
+sudo service pf stop        # shutdown firewall
 sudo certbot renew
-sudo service pf start  # start firewall
+sudo service pf start       # start firewall
 ```
  
-Again, copy certificate files `privkey.pem` and `cert.pem` to `~/.local/share/certs`.
+Again, copy certificate files `privkey.pem` and `cert.pem` to
+`~/.local/share/certs`.
  
 ---
  
 ## Upgrading
  
-From time to time there can be updates to perguntations, python packages and javascript libraries.
-
-Python packages can be upgraded independently of the rest using pip:
+From time to time there can be updates to perguntations, python packages and
+javascript libraries.  Python packages can be upgraded independently of the
+rest using pip:
  
 ```sh
-pip list --outdated         # lists upgradable packages
+pip list --outdated --user  # lists upgradable packages
 pip install -U something    # upgrade something
 ```
  
+Attention: don't upgrade blindly. Some upgrades can be incompatible with
+perguntations and break things! All upgrades require testing of all the
+funcionalities.
+
 To upgrade perguntations and javascript libraries do:
  
 ```sh
 cd perguntations
 git pull                    # get latest version of perguntations
 npm update                  # get latest versions of javascript libraries
-pip3 install -U .           # upgrade perguntations
+pip install -U .            # upgrade perguntations
 ```
  
 ## Troubleshooting
  
 - The server tries to run `python3` so this command must be accessible from
-user accounts. Currently, the minimum supported python version is 3.7.
-
+  user accounts. Currently, the minimum supported python version is 3.8.
 - If you are getting any `UnicodeEncodeError` type of errors that's because the
-terminal is not supporting UTF-8.
-This error may occur when a unicode character is printed to the screen by the
-server or, when running question generator or correction scripts, a message is
-piped between the server and the scripts that includes unicode characters.
-Try running `locale` on the terminal and see if there are any error messages.
-Solutions:
+  terminal is not supporting UTF-8.  This error may occur when a unicode
+  character is printed to the screen by the server or, when running question
+  generator or correction scripts, a message is piped between the server and
+  the scripts that includes unicode characters.  Try running `locale` on the
+  terminal and see if there are any error messages.
+  Solutions:
+
   - debian: `sudo dpkg-reconfigure locales` and select your UTF-8 locales.
   - FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
  
@@ -2,12 +2,12 @@
     "description": "Javascript libraries required to run the server",
     "email": "mjsb@uevora.pt",
     "dependencies": {
-        "@fortawesome/fontawesome-free": "^5.15.1",
-        "bootstrap": "^4.5",
-        "codemirror": "^5.58.1",
+        "@fortawesome/fontawesome-free": "^5.15.3",
+        "bootstrap": "^4.6.0",
+        "codemirror": "^5.61.1",
         "datatables": "^1.10",
-        "jquery": "^3.5.1",
-        "mathjax": "^3.1.2",
-        "underscore": "^1.11.0"
+        "jquery": "^3.6.0",
+        "mathjax": "^3.1.4",
+        "underscore": "^1.13.1"
     }
 }
...	...	@@ -11,26 +11,27 @@
11	11
12	12	## Requirements
13	13
14		-The webserver is a python application that requires `>=python3.7` and the package installer for python `pip`. The node package management `npm` is also necessary in order to install the javascript libraries.
	14	+The webserver is a python application that requires `>=python3.8` and the
	15	+package installer for python `pip`. The node package management `npm` is also
	16	+necessary in order to install the javascript libraries.
15	17
16	18	```sh
17	19	sudo apt install python3 python3-pip npm # Ubuntu
18		-sudo pkg install python37 py37-sqlite3 py37-pip npm # FreeBSD
19		-sudo port install python37 py37-pip py37-setuptools npm6 # MacOS (macports)
	20	+sudo pkg install python38 py38-sqlite3 py38-pip npm # FreeBSD
	21	+sudo port install python38 py38-pip py38-setuptools npm6 # MacOS (macports)
20	22	```
21	23
22		-To make the `pip` install packages to a local directory, the file `pip.conf` should be configured as follows:
	24	+To make the `pip` install packages to a local directory, the file `pip.conf`
	25	+should be configured as follows:
23	26
24	27	```ini
25	28	[global]
26	29	user=yes
27		-
28		-[list]
29		-format=columns
30	30	```
31	31
32	32	This file is usually in `~/.config/pip/` in Linux and FreeBSD. In MacOS it's in
33		-`~/Library/Application Support/pip/`. You may have to create it, if it doesn't exist yet.
	33	+`~/Library/Application Support/pip/`. You may have to create it, if it doesn't
	34	+exist yet.
34	35
35	36	---
36	37
...	...	@@ -42,19 +43,21 @@ Download and install:
42	43	git clone https://git.xdi.uevora.pt/mjsb/perguntations.git
43	44	cd perguntations
44	45	npm install
45		-pip3 install .
	46	+pip install .
46	47	```
47	48
48		-The command `npm` installs the javascript libraries and then `pip3` installs the python webserver. This will also install any required dependencies.
	49	+The command `npm` installs the javascript libraries and then `pip` installs the
	50	+python webserver. This will also install any required dependencies.
49	51
50		-Atention! `pip3 install .` must run after `npm install`, otherwise the javascript libraries will not be found during the install.
	52	+Atention! `pip install .` must be run after `npm install`, otherwise
	53	+the javascript libraries will not be found during the install.
51	54
52	55	---
53	56
54	57	## Setup
55	58
56		-The server will run a `https` server and requires valid certificates.
57		-There are two possibilities to generate the certificates:
	59	+The server will run a `https` server and requires valid certificates. There
	60	+are two possibilities to generate the certificates:
58	61
59	62	- public server with static IP address and registered domain name;
60	63	- private server on a local network isolated from the internet.
...	...	@@ -74,38 +77,35 @@ cd ~/.local/share/certs
74	77	openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes
75	78	```
76	79
77		-To have a true certificate, like the ones issued by LetsEncrypt, the host
78		-must have a registered domain name.
79		-See instructions for FreeBSD at the end of this page.
	80	+To have a true certificate, like the ones issued by LetsEncrypt, the host must
	81	+have a registered domain name. See instructions for FreeBSD at the end of this
	82	+page.
80	83
81	84	---
82	85
83	86	## Running a demo
84	87
85		-The directory `demo` in the repository includes a demo test that can be used
86		-as a template for your own tests and questions.
87		-
88		-To run the demonstration test you need to initialize the database using one of
89		-the following methods:
	88	+The directory `demo` in the repository includes a demo test that can be used as
	89	+a template for your own tests and questions. To run the demonstration test you
	90	+need to initialize the database using one of the following methods:
90	91
91	92	```sh
92	93	cd perguntations/demo
93		-
94	94	initdb students.csv # initialize from a CSV file
95	95	initdb --admin # only adds the administrator account
96	96	initdb --add 123 "Asterix Gaules" # add one student
97	97	```
98	98
99	99	This will create or update a `students.db` file that contains a sqlite3
100		-database.
101		-The database stores user passwords and grades, but not the actual tests.
	100	+database. The database stores user passwords and grades, but not the actual
	101	+tests.
102	102
103		-A test is specified in a single `yaml` file.
104		-The demo already includes the `demo.yaml` that you can play with.
	103	+A test is specified in a single `yaml` file. The file `demo.yaml` is a test
	104	+specification that you can play with.
105	105
106		-The complete tests submitted by the students are stored in JSON files in the
107		-directory defined in `demo.yaml` under the option `answers_dir: ans`.
108		-We also have to create this directory manually:
	106	+The answered tests submitted by the students are stored in JSON files in a
	107	+directory defined in `demo.yaml` under the option `answers_dir: ans`. We also
	108	+have to create this directory manually:
109	109
110	110	```sh
111	111	mkdir ans # directory where the tests will be saved
...	...	@@ -114,38 +114,36 @@ mkdir ans # directory where the tests will be saved
114	114	Start the server and run the `demo.yaml` test:
115	115
116	116	```sh
117		-perguntations demo.yaml # run demo test
	117	+perguntations demo.yaml # run demo test
118	118	```
119	119
120	120	Several options are available, run `perguntations --help` for a list.
121	121
122		-The server listens on port 8443 of all IPs of all network interfaces.
123		-Open the browser at `http://127.0.0.1:8443/` and login as user number `0`
	122	+The server listens on port 8443 of all IPs of all network interfaces. Open the
	123	+browser at `http://127.0.0.1:8443/` and login as user number `0`
124	124	(administrator) and choose any password you want. The password is defined on
125	125	first login.
	126	+
126	127	After logging in, you will be redirected to the administration page that shows
127	128	all the students and their current state.
128	129
129	130	1. Authorize students by clicking the checkboxes.
130	131	2. Open a different browser (or exit administrator) at `http://127.0.0.1:8443/`
131		-and login as one of the authorized students.
132		-Answer the questions and submit. You should get a grade at the end.
	132	+ and login as one of the authorized students. Answer the questions and
	133	+ submit. You should get a grade at the end.
133	134
134		-The server can be stoped from the terminal with `^C`.
	135	+The server can be stoped from the terminal with `^C` (Control+C).
135	136
136	137	---
137	138
138	139	## Running on lower ports
139	140
140	141	Ports 80 and 443 are reserved for the root user and and this software **should
141		-NOT be run as root**.
142		-Instead, tcp traffic can be forwarded from port 443 to 8443 where the server is
143		-listening.
144		-The details depend on the operating system/firewall.
145		-
146		-### debian
	142	+NOT be run as root**. Instead, tcp traffic can be forwarded from port 443 to
	143	+8443 where the server is listening. The details depend on the operating
	144	+system/firewall.
147	145
148		-FIXME: Untested
	146	+### debian (FIXME: Untested)
149	147
150	148	```.sh
151	149	iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-ports 8443
...	...	@@ -166,25 +164,29 @@ Explanation:
166	164
167	165	Edit `/etc/pf.conf`:
168	166
169		- ext_if="wlan1"
170		- rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
171		- rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
	167	+```conf
	168	+ext_if="wlan1"
	169	+rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
	170	+rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
	171	+```
172	172
173		-The `wlan1` should be the name of the network interface.
174		-Use `ext_if="vtnet0"` for guest additions under virtual box.
	173	+The `wlan1` should be the name of the network interface. Use `ext_if="vtnet0"`
	174	+for guest additions under virtual box.
175	175
176	176	Start firewall with `sudo service pf start`.
177	177
178	178	Optionally, to activate pf on boot, edit `rc.conf`:
179	179
180		- pf_enable="YES"
181		- pf_flags=""
182		- pf_rules="/etc/pf.conf"
	180	+```conf
	181	+pf_enable="YES"
	182	+pf_flags=""
	183	+pf_rules="/etc/pf.conf"
183	184
184		- # optional logging:
185		- pflog_enable="YES"
186		- pflog_flags=""
187		- pflog_logfile="/var/log/pflog"
	185	+# optional logging:
	186	+pflog_enable="YES"
	187	+pflog_flags=""
	188	+pflog_logfile="/var/log/pflog"
	189	+```
188	190
189	191	## Generating certificates with LetsEncript (FreeBSD)
190	192
...	...	@@ -192,7 +194,7 @@ Generating certificates for a public server (FreeBSD) requires a registered
192	194	domain with fixed IP.
193	195
194	196	```sh
195		-sudo pkg install py27-certbot # FreeBSD
	197	+sudo pkg install py38-certbot # FreeBSD 13
196	198	sudo service pf stop # disable pf firewall (FreeBSD)
197	199	sudo certbot certonly --standalone -d www.example.com
198	200	sudo service pf start # enable pf firewall
...	...	@@ -208,47 +210,52 @@ chmod 400 cert.pem privkey.pem
208	210	Certificate renewals can be done as follows:
209	211
210	212	```sh
211		-sudo service pf stop # shutdown firewall
	213	+sudo service pf stop # shutdown firewall
212	214	sudo certbot renew
213		-sudo service pf start # start firewall
	215	+sudo service pf start # start firewall
214	216	```
215	217
216		-Again, copy certificate files `privkey.pem` and `cert.pem` to `~/.local/share/certs`.
	218	+Again, copy certificate files `privkey.pem` and `cert.pem` to
	219	+`~/.local/share/certs`.
217	220
218	221	---
219	222
220	223	## Upgrading
221	224
222		-From time to time there can be updates to perguntations, python packages and javascript libraries.
223		-
224		-Python packages can be upgraded independently of the rest using pip:
	225	+From time to time there can be updates to perguntations, python packages and
	226	+javascript libraries. Python packages can be upgraded independently of the
	227	+rest using pip:
225	228
226	229	```sh
227		-pip list --outdated # lists upgradable packages
	230	+pip list --outdated --user # lists upgradable packages
228	231	pip install -U something # upgrade something
229	232	```
230	233
	234	+Attention: don't upgrade blindly. Some upgrades can be incompatible with
	235	+perguntations and break things! All upgrades require testing of all the
	236	+funcionalities.
	237	+
231	238	To upgrade perguntations and javascript libraries do:
232	239
233	240	```sh
234	241	cd perguntations
235	242	git pull # get latest version of perguntations
236	243	npm update # get latest versions of javascript libraries
237		-pip3 install -U . # upgrade perguntations
	244	+pip install -U . # upgrade perguntations
238	245	```
239	246
240	247	## Troubleshooting
241	248
242	249	- The server tries to run `python3` so this command must be accessible from
243		-user accounts. Currently, the minimum supported python version is 3.7.
244		-
	250	+ user accounts. Currently, the minimum supported python version is 3.8.
245	251	- If you are getting any `UnicodeEncodeError` type of errors that's because the
246		-terminal is not supporting UTF-8.
247		-This error may occur when a unicode character is printed to the screen by the
248		-server or, when running question generator or correction scripts, a message is
249		-piped between the server and the scripts that includes unicode characters.
250		-Try running `locale` on the terminal and see if there are any error messages.
251		-Solutions:
	252	+ terminal is not supporting UTF-8. This error may occur when a unicode
	253	+ character is printed to the screen by the server or, when running question
	254	+ generator or correction scripts, a message is piped between the server and
	255	+ the scripts that includes unicode characters. Try running `locale` on the
	256	+ terminal and see if there are any error messages.
	257	+ Solutions:
	258	+
252	259	- debian: `sudo dpkg-reconfigure locales` and select your UTF-8 locales.
253	260	- FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
254	261
...	...
...	...	@@ -2,12 +2,12 @@
2	2	"description": "Javascript libraries required to run the server",
3	3	"email": "mjsb@uevora.pt",
4	4	"dependencies": {
5		- "@fortawesome/fontawesome-free": "^5.15.1",
6		- "bootstrap": "^4.5",
7		- "codemirror": "^5.58.1",
	5	+ "@fortawesome/fontawesome-free": "^5.15.3",
	6	+ "bootstrap": "^4.6.0",
	7	+ "codemirror": "^5.61.1",
8	8	"datatables": "^1.10",
9		- "jquery": "^3.5.1",
10		- "mathjax": "^3.1.2",
11		- "underscore": "^1.11.0"
	9	+ "jquery": "^3.6.0",
	10	+ "mathjax": "^3.1.4",
	11	+ "underscore": "^1.13.1"
12	12	}
13	13	}
...	...