Commit 0057db016de7e158802129f03427afbb711ef777

Authored by Miguel Barão
1 parent 7e584184
Exists in master and in 1 other branch dev

- documentation update

Showing 3 changed files with 23 additions and 376 deletions   Show diff stats
@@ -2,11 +2,12 @@ @@ -2,11 +2,12 @@
2 # BUGS 2 # BUGS
3 3
4 - se directorio logs não existir no directorio actual (não perguntations) rebenta. 4 - se directorio logs não existir no directorio actual (não perguntations) rebenta.
5 -- usar thread.Lock para aceder a variaveis de estado?  
6 -- servidor nao esta a lidar com eventos scroll/resize  
7 5
8 # TODO 6 # TODO
9 7
  8 +- SSL
  9 +- usar thread.Lock para aceder a variaveis de estado?
  10 +- servidor nao esta a lidar com eventos scroll/resize
10 - se ocorrer um erro na correcçao avisar aluno para contactar o professor. 11 - se ocorrer um erro na correcçao avisar aluno para contactar o professor.
11 - implementar practice mode. 12 - implementar practice mode.
12 - usar http://wtfforms.com para radio e checkboxes 13 - usar http://wtfforms.com para radio e checkboxes
@@ -6,14 +6,12 @@ After installing python and the required packages as described in the `README.md @@ -6,14 +6,12 @@ After installing python and the required packages as described in the `README.md
6 6
7 ``` 7 ```
8 $ cd PATH/TO/perguntations 8 $ cd PATH/TO/perguntations
9 -$ ./initdb.py 9 +$ ./initdb.py --demo
10 New database created: students.db 10 New database created: students.db
11 10 users inserted: 11 10 users inserted:
12 0 - Professor (administrator) 12 0 - Professor (administrator)
13 1465 - Gil Vicente 13 1465 - Gil Vicente
14 - .  
15 - .  
16 - . 14 + ... - ...
17 1924 - Alexandre O'Neill 15 1924 - Alexandre O'Neill
18 $ mv students.db demo/ 16 $ mv students.db demo/
19 $ ./serve.py demo/test.yaml 17 $ ./serve.py demo/test.yaml
@@ -28,16 +26,15 @@ $ ./serve.py demo/test.yaml @@ -28,16 +26,15 @@ $ ./serve.py demo/test.yaml
28 26
29 You can now open a browser and point it to the server address, e.g. `http://127.0.0.1:8080`. 27 You can now open a browser and point it to the server address, e.g. `http://127.0.0.1:8080`.
30 28
31 -Enter `0` for the number and any password. This will be your password from now on. 29 +Enter `0` for the number and choose any password. This will be your password from now on.
32 30
33 After login, you will see an administrator page. This page allows you to: 31 After login, you will see an administrator page. This page allows you to:
34 32
35 -- Generate a sample test just for you to see if it looks right.  
36 - Allow students to login to the test. This can be done one at a time using the checkboxes on the left of each student, or allow all students by clicking the button in the bottom of the page. 33 - Allow students to login to the test. This can be done one at a time using the checkboxes on the left of each student, or allow all students by clicking the button in the bottom of the page.
37 - Review a test done by a student by clicking the grade colorbar on the right. If a student has multiple tests, then several bars will be shown. A tooltip shows the date/time for each test. 34 - Review a test done by a student by clicking the grade colorbar on the right. If a student has multiple tests, then several bars will be shown. A tooltip shows the date/time for each test.
38 - Reset student passwords. Students that already have a password will have a label `PW` next to the student name. To reset the password, go to the bottom of the page and insert the student number in the box and press the button to reset the password. The label `PW` will disappear. 35 - Reset student passwords. Students that already have a password will have a label `PW` next to the student name. To reset the password, go to the bottom of the page and insert the student number in the box and press the button to reset the password. The label `PW` will disappear.
39 -- Online students will be marked and the table on the top of the page will show some more information (login time, ip address, etc)  
40 -- If a student moves away from the browser (e.g. for cheating), a red bar appears along with the status `unfocus`. Screen saver is also reported as unfocus, so some care should be taken... 36 +- Online students will be highlighted and the table on the top of the page will show some more information (login time, ip address, etc)
  37 +- If a student moves unfocus the browser window on his computer (e.g. for cheating), a red bar appears along with the status `unfocus`. Screen saver is also reported as unfocus, so some care should be taken...
41 - It is also possible to add new students by clicking the button on the bottom of the page. 38 - It is also possible to add new students by clicking the button on the bottom of the page.
42 39
43 ## Answering a test 40 ## Answering a test
@@ -50,380 +47,29 @@ After answering all the questions, the test is submited by clicking the button o @@ -50,380 +47,29 @@ After answering all the questions, the test is submited by clicking the button o
50 47
51 ## Creating a new test 48 ## Creating a new test
52 49
53 -A test is just a yaml file with the configuration for that test. Look at `demo/test.yaml` for an example. 50 +A test is specified in a single yaml file. Look at `demo/test.yaml` for an example and documentation.
54 51
55 ## Creating questions 52 ## Creating questions
56 53
57 -Questions are defined in yaml files. Each yaml file contains a list of questions 54 +Questions are defined in yaml files. Each yaml file contains a list of questions from which some can be chosen to be part of a test.
58 55
  56 +The types of questions supported are:
59 57
  58 +- `radio` questions are multiple choice where only one can be selected.
  59 +- `checkbox` questions are also multiple choice but several can be selected.
  60 +- `text` and `text_regex` questions provide a line to write the answer.
  61 +- `textarea` provide a multiline text box to write the answer (e.g. for computer code). The answer is corrected by calling an external program that is not part of the server and is up to you to right it.
60 62
  63 +Besides these, there are `information` and `alert` types which just show text but do not expect answers.
61 64
  65 +All the types above are predifined questions. It is also possible to generate questions by code. These are defined as type `generator` and an external program must be provided to generate a question of the types above.
62 66
  67 +An example file with these types of questions is provided in `demo/questions/questions.yaml` along with accompanying files under the directory `demo/questions`.
63 68
64 69
  70 +## Testing a new test
65 71
  72 +Before using a test in product you should test is very carefully. After writing the questions and the test configuration, my recommendation is to:
66 73
67 -  
68 -  
69 -  
70 -  
71 -  
72 -  
73 -  
74 -Before using the program you need to  
75 -  
76 -1. Edit `config/server.conf` in the server directory and define  
77 - - Logging  
78 -  
79 - `log.error_file= '/Users/USERNAME/Library/Logs/Perguntations/errors.log'`  
80 -  
81 - `log.access_file= '/Users/USERNAME/Library/Logs/Perguntations/access.log'`  
82 -  
83 - You must create the directories if they do not exist already.  
84 -  
85 - Setting these locations to empty strings `''` disables logging.  
86 -  
87 - - Sessions  
88 - If `tools.sessions.storage_type='file'` sessions are saved on the file system in the location given in `tools.sessions.storage_path`. Restarting the server will maintain the sessions active.  
89 -  
90 - If `storage_type='ram'` (default) no files are stored but restaring the server will reset sessions.  
91 -  
92 - You should give enough time in the `tools.sessions.timeout` to complete an exam. The default is 240 minutes (4 hours).  
93 -  
94 -1. Create the students database (see below)  
95 -1. Create questions (see below)  
96 -1. Create a test (see below)  
97 -  
98 -### Create students database  
99 -  
100 -We need a sqlite3 database to store students, passwords, test results, and questions results, etc.  
101 -  
102 -The database can be initialized from a list of students in CSV format by running in the terminal  
103 -  
104 -```sh  
105 -./initdb_from_csv.py list_of_students.csv  
106 -```  
107 -  
108 -This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords.  
109 -It also adds a special user number 0. This is the administrator user (Professor).  
110 -  
111 -The passwords will be defined on the first login.  
112 -  
113 -### Create new questions  
114 -  
115 -Questions are defined in `yaml` files and can reside anywhere in the filesystem.  
116 -Each file contains a list of questions, where each question is a dictionary. Example  
117 -  
118 -```yaml  
119 --  
120 - ref: question-1  
121 - type: radio  
122 - text: Select the correct option  
123 - options:  
124 - - correct  
125 - - wrong  
126 -  
127 --  
128 - ref: question-2  
129 - type: checkbox  
130 - text: Which ones are correct?  
131 - options:  
132 - - correct  
133 - - correct  
134 - - wrong  
135 - correct: [1, 1, -1]  
136 - hint: There are two correct answers!  
137 -```  
138 -  
139 -There are several kinds of questions:  
140 -  
141 -- __information__: nothing to answer  
142 -- __radio__: only one option is correct  
143 -- __checkbox__: several options are correct  
144 -- __text__: compares text with a list of accepted answers  
145 -- __text_regex__: matches text agains regular expression  
146 -- __textarea__: send text to an external script for validation  
147 -- __generator__: the question is generated from an external script, the actual question generated can be any of the above types.  
148 -  
149 -Detailed information on each question type is described later on.  
150 -  
151 -  
152 -### Creating a new test  
153 -  
154 -A test is a file in `yaml` format that can reside anywhere on the filesystem. It has the following structure:  
155 -  
156 -```yaml  
157 -ref: this-is-a-key  
158 -title: Titulo do teste  
159 -database: db/mystudents.db  
160 -  
161 -# Will save the entire test of each student in JSON format.  
162 -# If tests are to be saved, we must specify the directory.  
163 -# The directory is created if it doesn't exist already.  
164 -# The name of the JSON files will include the student number, test  
165 -# reference key, date and time.  
166 -save_answers: True  
167 -answers_dir: ans/asc1_test4  
168 -  
169 -# Some questions can contain hints, embedded videos, etc  
170 -show_hints: True  
171 -  
172 -# Each question has some number of points. Show them normalized to 0-20.  
173 -show_points: True  
174 -  
175 -# In train mode, the correction of the test is shown and the test can  
176 -# be repeated  
177 -practice_mode: True  
178 -  
179 -# Show the data structures obtained from the test and the questions  
180 -debug: False  
181 -  
182 -# Show the file and ref field of each question  
183 -show_ref: True  
184 -  
185 -# ----------------------------------------------------------------------------  
186 -# Location of the questions files (absolute path or relative to current dir)  
187 -path: questions  
188 -  
189 -# This are the questions files to be imported.  
190 -files:  
191 - - file1.yaml  
192 - - file2.yaml  
193 - - file3.yaml  
194 -  
195 -# ----------------------------------------------------------------------------  
196 -# This is the actual test configuration. Selection of questions and points  
197 -# It'a defined as a list of questions. Each question can be a single  
198 -# question key or a list of keys from which one is chosen at random.  
199 -# Each question has a default value of 1.0 point, but it can be overridden.  
200 -# The points defined here do not need to be normalized (it's automatic).  
201 -questions:  
202 - - ref:  
203 - - first-question-1 # randomly choose one from these 3 questions  
204 - - first-question-2  
205 - - first-question-3  
206 - points: 0.5  
207 -  
208 - - ref: second-question # one question, 1.0 point (unnormalized)  
209 -  
210 - - third-question # "ref:" not needed in simple cases  
211 -  
212 -This following one is wrong:  
213 -  
214 - - wrong-question # missing "ref:" key  
215 - points: 2  
216 -```  
217 -  
218 -Some of the options have default values if they are omitted. The defaults are the following:  
219 -  
220 -```yaml  
221 -ref: filename.yaml  
222 -title: ''  
223 -save_answers: False  
224 -show_hints: False  
225 -show_points: False  
226 -practice_mode: False  
227 -show_ref: False  
228 -debug: False  
229 -points: 1.0  
230 -```  
231 -### Running an existing test  
232 -  
233 -A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument:  
234 -  
235 -```sh  
236 -$ ./serve.py tests_dir/mytest.yaml  
237 -```  
238 -  
239 -Some defaults can be overriden with command line options. Example  
240 -  
241 -```sh  
242 -$ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers  
243 -```  
244 -To terminate the test just do `^C` on the keyboard.  
245 -  
246 -## Questions  
247 -  
248 -Every question should have a `ref` and a `type`. The other keys depend on the type of question.  
249 -  
250 -### Information  
251 -  
252 -Not a real question. Just text to be shown without expecting an answer.  
253 -  
254 -```yaml  
255 --  
256 - ref: some-key  
257 - type: information  
258 - text: Tomorrow will rain.  
259 -```  
260 -Correcting an information will always be considered correct, but the grade will be zero because it has 0.0 points by default.  
261 -  
262 -### Radio  
263 -  
264 -Only one option is correct.  
265 -  
266 -```yaml  
267 --  
268 - ref: some-key  
269 - type: radio  
270 - text: The horse is white. # optional (default: '')  
271 - options:  
272 - - The horse is white  
273 - - The horse is not black  
274 - - The horse is black  
275 - correct: 0 # optional (default: 0). Index is 0-based.  
276 - shuffle: True # optional (default: True)  
277 - discount: True # optional (default: True)  
278 -```  
279 -The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`.  
280 -  
281 -Wrong answers discount by default. If there are half-right answers, the discount values are calculated automatically. `discount: False` disables the discount calculation and the values are the ones defined in `correct`.  
282 -  
283 -### Checkbox  
284 -  
285 -There can be several options correct. Each option is like answering an independent question.  
286 -  
287 -```yaml  
288 --  
289 - ref: some-key  
290 - type: checkbox  
291 - text: The horse is white. # optional (default: '')  
292 - options:  
293 - - The horse is white  
294 - - The horse is not black  
295 - - The horse is black  
296 - correct: [1,1,-1] # optional (default: [0,0,0]).  
297 - shuffle: True # optional (default: True)  
298 - discount: True # optional (default: True)  
299 -```  
300 -Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value.  
301 -E.g. consider `correct: [1, 0.5, -1]`, then  
302 -  
303 -- if the first option is marked the value is 1, otherwise if it's unmarked the value is -1.  
304 -- if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5.  
305 -- if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one)  
306 -  
307 -`discount: False` disables the discount and the values are the ones defined in `correct` if the answer is right, or 0.0 if wrong.  
308 -  
309 -### Text  
310 -  
311 -The answer is a line of text.  
312 -The server will check if the answer exactly matches the correct one.  
313 -  
314 -```yaml  
315 --  
316 - ref: some-key  
317 - type: text  
318 - text: What's your favorite color? # optional (default: '')  
319 - correct: white  
320 -```  
321 -alternatively, we can give a list of acceptable answers  
322 -```yaml  
323 - correct: ['white', 'blue', 'red']  
324 -```  
325 -### Regular expression  
326 -  
327 -The answer is a line of text.  
328 -The server will check if the answer matches a regular expression.  
329 -  
330 -```yaml  
331 --  
332 - ref: some-key  
333 - type: text_regex  
334 - text: What's your favorite color? # optional (default: '')  
335 - correct: '[Ww]hite'  
336 -```  
337 -  
338 -Careful: yaml does not support raw text. Some characters have to be escaped.  
339 -  
340 -### Text area  
341 -  
342 -The answer is given in a textarea. The text (usually code) is sent to an external program running on a separate process for validation.  
343 -The external program should accept input from stdin, and print to stdout a single number in the interval 0.0 to 1.0 indicating the level of correctness.  
344 -The server will try to convert the printed message to a float, a failure will give 0.0.  
345 -  
346 -```yaml  
347 --  
348 - ref: some-key  
349 - type: textarea  
350 - text: write an expression to add x and y. # optional (default: '')  
351 - correct: myscript  
352 - # optional  
353 - lines: 15  
354 -```  
355 -  
356 -The script location is the same as the questions file.  
357 -An example of a script in python that validades an answer is  
358 -  
359 -```python  
360 -#!/usr/bin/env python3.4  
361 -  
362 -import sys  
363 -s = sys.stdin.read()  
364 -if s == 'Alibaba':  
365 - print(1.0)  
366 -else:  
367 - print(0.0)  
368 -exit(0)  
369 -```  
370 -  
371 -but any script language or executable program can be used for this purpose.  
372 -  
373 -  
374 -### Generator  
375 -  
376 -A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!).  
377 -  
378 -```yaml  
379 --  
380 - ref: some-key  
381 - type: generator  
382 - script: path/to/generator_script  
383 - # arg: "optional string passed on to stdin of the script"  
384 -```  
385 -  
386 -An example of a question generator is the following  
387 -  
388 -```python  
389 -#!/usr/bin/env python3.4  
390 -from random import randint  
391 -import sys  
392 -  
393 -# read arguments from stdin and convert to integers  
394 -arg = sys.stdin.read()  
395 -a,b = (int(n) for n in arg.split(','))  
396 -  
397 -# generate question  
398 -x = randint(a, b)  
399 -y = randint(a, b)  
400 -s = '''  
401 -ref: addition  
402 -type: text  
403 -text: How much is {0} plus {1}?  
404 -correct: {2}  
405 -'''.format(x, y, x + y)  
406 -  
407 -# send question to stdout  
408 -print(s)  
409 -```  
410 -  
411 -## Writing good looking questions  
412 -  
413 -The text of the questions (and options in radio and checkbox type questios) is parsed as markdown and code is prettyfied using Pygments. Equations can be inserted like in LaTeX and are rendered using MathJax.  
414 -  
415 -A good way to define multiple lines of text in the questions is to use the bar |. Yaml will use all the text that is indented to the right of that column. Example  
416 -  
417 -```yaml  
418 - text: |  
419 - Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX  
420 - and pretty code in several languages  
421 -  
422 - ```.C  
423 - int main(){  
424 - return 0;  
425 - }  
426 - ```  
427 - # this line stops the text because it is not indented  
428 -```  
429 - 74 +1. `./serve.py --allow-all shiny_new_test.yaml` where the `--allow-all` option avoids having to use a second browser for the admin page to explicitly allow a student.
  75 +2. If bugs are found, try commenting questions in the `shiny_new_test.yaml` to isolate the bug. Use the `--debug` option to add more information in the webpage.
@@ -121,7 +121,7 @@ class StudentWebService(object): @@ -121,7 +121,7 @@ class StudentWebService(object):
121 121
122 @cherrypy.tools.accept(media='application/json') # FIXME 122 @cherrypy.tools.accept(media='application/json') # FIXME
123 def POST(self, **args): 123 def POST(self, **args):
124 - uid = cherrypy.session.get(SESSION_KEY) 124 + # uid = cherrypy.session.get(SESSION_KEY)
125 if args['cmd'] == 'focus': 125 if args['cmd'] == 'focus':
126 v = json.loads(args['value']) 126 v = json.loads(args['value'])
127 self.app.set_student_focus(uid=args['number'], value=v) 127 self.app.set_student_focus(uid=args['number'], value=v)