- Modified how paths to scripts are handled. Paths are now relative to the direc…

…tory of the question yaml file (instead of questions_dir).

- Modified how paths to scripts are handled. Paths are now relative to the direc…
…tory of the question yaml file (instead of questions_dir).
Miguel Barão
1 parent 91893722
Showing 4 changed files with 67 additions and 21 deletions Show diff stats
BUGS.md
README.md
questions.py
serve.py
@@ -2,13 +2,12 @@
 # BUGS
  
 - Review de um teste que foi apagado rebenta.
-- Gerar pdf's com todos os testes no final (pdfkit).
 - permitir eliminar teste a decorrer de modo a que o aluno possa recomeçar (e.g. noutro browser)
 - servidor nao esta a lidar com eventos scroll/resize. ignorar?
  
 # TODO
  
-- configurar pf em freebsd, port forward 80 -> 8080. documentacao
+- Gerar pdf's com todos os testes no final (pdfkit).
 - testar SSL
 - manter registo dos unfocus durante o teste e de qual a pergunta visivel nesse momento
  
@@ -28,6 +27,8 @@
  
 # FIXED
  
+- configuração do teste não joga bem com o do aprendizations. Em particular os scripts não ficam com o mesmo path!!!
+- configurar pf em freebsd, port forward 80 -> 8080. documentacao
 - barras com notas em grade estão desalinhadas.
 - erros nos generators devem ser ERROR e não WARNING.
 - se directorio "logs" não existir no directorio actual aborta com mensagem de erro.
@@ -8,7 +8,7 @@
  
 ### Requirements
  
-The webserver is a python application and requires `python3.5` and `pip` to be installed, plus the following additional packages:
+The webserver is a python application and requires `python3.6` and `pip` to be installed, plus the following additional packages:
  
 - CherryPy
 - Mako
@@ -29,7 +29,7 @@ pip install --user cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt
 #### Installing packages in a virtual environment (alternative)
  
 ```.bash
-pyvenv-3.5 venv/perguntations              # or other virtualenv directory
+pyvenv-3.6 venv/perguntations              # or other virtualenv directory
 source venv/perguntations/bin/activate     # activate virtualenv
  
 pip install cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt
@@ -80,7 +80,7 @@ We are now ready to run the server:
 ./serve.py demo/test.yaml    # run demo test
 ```
  
-By default the server listens on all IPs of all network interfaces.
+By default the server listens on port 8080 and on all IPs of all network interfaces.
 Open the browser at `http://127.0.0.1:8080/` and login as user number `0` (administrator) and choose any password. Then
  
 1. Authorize students by clicking the checkboxes.
@@ -90,7 +90,7 @@ The server can be stoped from the terminal with `^C`.
  
 ## Running on port 80
  
-Port 80 is reserved for the root user and and this software _should NOT be run as root_. Instead, the firewall should be configured to forward tcp traffic from port 80 to 8080 where the server is listening. The details depend on the operating system.
+Ports 80 and 443 are reserved for the root user and and this software _should NOT be run as root_. Instead, the firewall should be configured to forward tcp traffic from port 80 to 8080 where the server is listening. The details depend on the operating system.
  
 ### debian:
  
@@ -109,9 +109,28 @@ Explanation:
 - `-j REDIRECT` what to do when packet matches the rule.
 - `--to-ports 8080` where to redirect the packets.
  
-### freebsd
+### FreeBSD and pf
  
-Todo...
+Edit `/etc/pf.conf`:
+
+    ext_if="em0"
+    rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
+    rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
+
+or `ext_if="vtnet0"` for guest additions under virtual box.
+
+Edit `rc.conf`
+
+    pf_enable="YES"
+    pf_flags=""
+    pf_rules="/etc/pf.conf"
+
+    # optional logging:
+    pflog_enable="YES"
+    pflog_flags=""
+    pflog_logfile="/var/log/pflog"
+
+Reboot.
  
 ## Enabling SSL
  
@@ -119,12 +138,20 @@ Todo...
  
 ## Troubleshooting
  
-* The server tries to run `python3`. If only a `python3.5` command is available, you need to set the default python using the OS package manager or manually create a symbolic link:
-    - macOS/macports: `sudo port select --set python3 python35`
-    - FreeBSD 11: `cd /usr/local/bin; ln -s python3.5 python3`
+* The server tries to run `python3`. If only a `python3.6` command is available, you need to set the default python using the OS package manager or manually create a symbolic link:
+    - macOS/macports: `sudo port select --set python3 python36`
+    - FreeBSD 11: `cd /usr/local/bin; ln -s python3.6 python3`
  
-* If you are getting any `UnicodeEncodeError` type of errors that's because the terminal is not supporting UTF-8. Try running `locale` on the terminal and see if there is any error messages. Solutions:
+* If you are getting any `UnicodeEncodeError` type of errors that's because the terminal is not supporting UTF-8. This error may occur when a unicode character is printed to the screen by the server or, when running question generator or correction scripts, a message is piped between the server and the scripts that includes unicode characters.
+Try running `locale` on the terminal and see if there are any error messages. Solutions:
     - debian: fix it with `sudo dpkg-reconfigure locales`, select your UTF-8 locales and try again.
+    - FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
+
+        ```
+        me:\
+                :charset=UTF-8:\
+                :lang=en_US.UTF-8:
+        ```
  
 ## Contribute ###
  
@@ -83,24 +83,33 @@ class QuestionFactory(dict):
     # -----------------------------------------------------------------------
     # load single YAML questions file
     # -----------------------------------------------------------------------
-    def load_file(self, filename, questions_dir=''):
-        f = path.normpath(path.join(questions_dir, filename))
-        questions = load_yaml(f, default=[])
+    def load_file(self, pathfile, questions_dir=''):
+        # questions_dir is a base directory
+        # pathfile is a path of a file under the questions_dir
+        # For example, if
+        #    pathfile = 'math/questions.yaml'
+        #    questions_dir = '/home/john/questions'
+        # then the complete path is
+        #    fullpath = '/home/john/questions/math/questions.yaml'
+        fullpath = path.normpath(path.join(questions_dir, pathfile))
+        (dirname, filename) = path.split(fullpath)
+
+        questions = load_yaml(fullpath, default=[])
  
         n = 0
         for i, q in enumerate(questions):
             if isinstance(q, dict):
                 q.update({
                     'filename': filename,
-                    'path': questions_dir,
+                    'path': dirname,
                     'index': i                # position in the file, 0 based
                     })
                 self.add(q)                   # add question
                 n += 1                        # counter
             else:
-                logger.error('Question index {0} from file {1} is not a dictionary. Skipped!'.format(i, filename))
+                logger.error('Question index {0} from file {1} is not a dictionary. Skipped!'.format(i, pathfile))
  
-        logger.info('Loaded {0} questions from "{1}".'.format(n, filename))
+        logger.info('Loaded {0} questions from "{1}".'.format(n, pathfile))
  
     # -----------------------------------------------------------------------
     # load multiple YAML question files
 #!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-
  
 from os import path
 import sys
@@ -9,10 +7,21 @@ import logging.config
 import json
  
 try:
+    # required here
     import cherrypy
     from mako.lookup import TemplateLookup
+    # required elsewhere
+    from json import __version__ as json_version
+    from bcrypt import __version__ as bcrypt_version
+    from sqlalchemy import __version__ as alchemy_version
+    from yaml import __version__ as yaml_version
+    from markdown import version as markdown_version
 except ImportError:
-    print('Some python packages are missing. See README.md for instructions.')
+    print('''
+        Some python packages are missing.
+        Try running "pip3 install --user cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt"
+        See README.md for instructions.
+        ''')
     sys.exit(1)
  
 from tools import load_yaml
...	...	@@ -2,13 +2,12 @@
2	2	# BUGS
3	3
4	4	- Review de um teste que foi apagado rebenta.
5		-- Gerar pdf's com todos os testes no final (pdfkit).
6	5	- permitir eliminar teste a decorrer de modo a que o aluno possa recomeçar (e.g. noutro browser)
7	6	- servidor nao esta a lidar com eventos scroll/resize. ignorar?
8	7
9	8	# TODO
10	9
11		-- configurar pf em freebsd, port forward 80 -> 8080. documentacao
	10	+- Gerar pdf's com todos os testes no final (pdfkit).
12	11	- testar SSL
13	12	- manter registo dos unfocus durante o teste e de qual a pergunta visivel nesse momento
14	13
...	...	@@ -28,6 +27,8 @@
28	27
29	28	# FIXED
30	29
	30	+- configuração do teste não joga bem com o do aprendizations. Em particular os scripts não ficam com o mesmo path!!!
	31	+- configurar pf em freebsd, port forward 80 -> 8080. documentacao
31	32	- barras com notas em grade estão desalinhadas.
32	33	- erros nos generators devem ser ERROR e não WARNING.
33	34	- se directorio "logs" não existir no directorio actual aborta com mensagem de erro.
...	...
...	...	@@ -8,7 +8,7 @@
8	8
9	9	### Requirements
10	10
11		-The webserver is a python application and requires `python3.5` and `pip` to be installed, plus the following additional packages:
	11	+The webserver is a python application and requires `python3.6` and `pip` to be installed, plus the following additional packages:
12	12
13	13	- CherryPy
14	14	- Mako
...	...	@@ -29,7 +29,7 @@ pip install --user cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt
29	29	#### Installing packages in a virtual environment (alternative)
30	30
31	31	```.bash
32		-pyvenv-3.5 venv/perguntations # or other virtualenv directory
	32	+pyvenv-3.6 venv/perguntations # or other virtualenv directory
33	33	source venv/perguntations/bin/activate # activate virtualenv
34	34
35	35	pip install cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt
...	...	@@ -80,7 +80,7 @@ We are now ready to run the server:
80	80	./serve.py demo/test.yaml # run demo test
81	81	```
82	82
83		-By default the server listens on all IPs of all network interfaces.
	83	+By default the server listens on port 8080 and on all IPs of all network interfaces.
84	84	Open the browser at `http://127.0.0.1:8080/` and login as user number `0` (administrator) and choose any password. Then
85	85
86	86	1. Authorize students by clicking the checkboxes.
...	...	@@ -90,7 +90,7 @@ The server can be stoped from the terminal with `^C`.
90	90
91	91	## Running on port 80
92	92
93		-Port 80 is reserved for the root user and and this software _should NOT be run as root_. Instead, the firewall should be configured to forward tcp traffic from port 80 to 8080 where the server is listening. The details depend on the operating system.
	93	+Ports 80 and 443 are reserved for the root user and and this software _should NOT be run as root_. Instead, the firewall should be configured to forward tcp traffic from port 80 to 8080 where the server is listening. The details depend on the operating system.
94	94
95	95	### debian:
96	96
...	...	@@ -109,9 +109,28 @@ Explanation:
109	109	- `-j REDIRECT` what to do when packet matches the rule.
110	110	- `--to-ports 8080` where to redirect the packets.
111	111
112		-### freebsd
	112	+### FreeBSD and pf
113	113
114		-Todo...
	114	+Edit `/etc/pf.conf`:
	115	+
	116	+ ext_if="em0"
	117	+ rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
	118	+ rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
	119	+
	120	+or `ext_if="vtnet0"` for guest additions under virtual box.
	121	+
	122	+Edit `rc.conf`
	123	+
	124	+ pf_enable="YES"
	125	+ pf_flags=""
	126	+ pf_rules="/etc/pf.conf"
	127	+
	128	+ # optional logging:
	129	+ pflog_enable="YES"
	130	+ pflog_flags=""
	131	+ pflog_logfile="/var/log/pflog"
	132	+
	133	+Reboot.
115	134
116	135	## Enabling SSL
117	136
...	...	@@ -119,12 +138,20 @@ Todo...
119	138
120	139	## Troubleshooting
121	140
122		-* The server tries to run `python3`. If only a `python3.5` command is available, you need to set the default python using the OS package manager or manually create a symbolic link:
123		- - macOS/macports: `sudo port select --set python3 python35`
124		- - FreeBSD 11: `cd /usr/local/bin; ln -s python3.5 python3`
	141	+* The server tries to run `python3`. If only a `python3.6` command is available, you need to set the default python using the OS package manager or manually create a symbolic link:
	142	+ - macOS/macports: `sudo port select --set python3 python36`
	143	+ - FreeBSD 11: `cd /usr/local/bin; ln -s python3.6 python3`
125	144
126		-* If you are getting any `UnicodeEncodeError` type of errors that's because the terminal is not supporting UTF-8. Try running `locale` on the terminal and see if there is any error messages. Solutions:
	145	+* If you are getting any `UnicodeEncodeError` type of errors that's because the terminal is not supporting UTF-8. This error may occur when a unicode character is printed to the screen by the server or, when running question generator or correction scripts, a message is piped between the server and the scripts that includes unicode characters.
	146	+Try running `locale` on the terminal and see if there are any error messages. Solutions:
127	147	- debian: fix it with `sudo dpkg-reconfigure locales`, select your UTF-8 locales and try again.
	148	+ - FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
	149	+
	150	+ ```
	151	+ me:\
	152	+ :charset=UTF-8:\
	153	+ :lang=en_US.UTF-8:
	154	+ ```
128	155
129	156	## Contribute ###
130	157
...	...
...	...	@@ -83,24 +83,33 @@ class QuestionFactory(dict):
83	83	# -----------------------------------------------------------------------
84	84	# load single YAML questions file
85	85	# -----------------------------------------------------------------------
86		- def load_file(self, filename, questions_dir=''):
87		- f = path.normpath(path.join(questions_dir, filename))
88		- questions = load_yaml(f, default=[])
	86	+ def load_file(self, pathfile, questions_dir=''):
	87	+ # questions_dir is a base directory
	88	+ # pathfile is a path of a file under the questions_dir
	89	+ # For example, if
	90	+ # pathfile = 'math/questions.yaml'
	91	+ # questions_dir = '/home/john/questions'
	92	+ # then the complete path is
	93	+ # fullpath = '/home/john/questions/math/questions.yaml'
	94	+ fullpath = path.normpath(path.join(questions_dir, pathfile))
	95	+ (dirname, filename) = path.split(fullpath)
	96	+
	97	+ questions = load_yaml(fullpath, default=[])
89	98
90	99	n = 0
91	100	for i, q in enumerate(questions):
92	101	if isinstance(q, dict):
93	102	q.update({
94	103	'filename': filename,
95		- 'path': questions_dir,
	104	+ 'path': dirname,
96	105	'index': i # position in the file, 0 based
97	106	})
98	107	self.add(q) # add question
99	108	n += 1 # counter
100	109	else:
101		- logger.error('Question index {0} from file {1} is not a dictionary. Skipped!'.format(i, filename))
	110	+ logger.error('Question index {0} from file {1} is not a dictionary. Skipped!'.format(i, pathfile))
102	111
103		- logger.info('Loaded {0} questions from "{1}".'.format(n, filename))
	112	+ logger.info('Loaded {0} questions from "{1}".'.format(n, pathfile))
104	113
105	114	# -----------------------------------------------------------------------
106	115	# load multiple YAML question files
...	...
1	1	#!/usr/bin/env python3
2		-# -- coding: utf-8 --
3		-
4	2
5	3	from os import path
6	4	import sys
...	...	@@ -9,10 +7,21 @@ import logging.config
9	7	import json
10	8
11	9	try:
	10	+ # required here
12	11	import cherrypy
13	12	from mako.lookup import TemplateLookup
	13	+ # required elsewhere
	14	+ from json import __version__ as json_version
	15	+ from bcrypt import __version__ as bcrypt_version
	16	+ from sqlalchemy import __version__ as alchemy_version
	17	+ from yaml import __version__ as yaml_version
	18	+ from markdown import version as markdown_version
14	19	except ImportError:
15		- print('Some python packages are missing. See README.md for instructions.')
	20	+ print('''
	21	+ Some python packages are missing.
	22	+ Try running "pip3 install --user cherrypy mako markdown pyyaml pygments sqlalchemy bcrypt"
	23	+ See README.md for instructions.
	24	+ ''')
16	25	sys.exit(1)
17	26
18	27	from tools import load_yaml
...	...