- replaced key 'stdin' by 'arg' in questions of type generator

- updated manual and demo

- replaced key 'stdin' by 'arg' in questions of type generator
- updated manual and demo
Miguel Barao
1 parent 9bace4f6
Showing 5 changed files with 224 additions and 181 deletions Show diff stats
BUGS.md
MANUAL.md
demo/generate-question.py
demo/questions.yaml
questions.py
@@ -7,7 +7,6 @@
 - parece que é preciso criar à mão a pasta para as respostas (ans/...) depois apercebo-me que os caminhos no teste dizem respeito à directoria donde o teste é corrido... as respostas deveriam guardadas no directório dado.
 - se database for mal configurada, é criada uma base de dados vazia e rebenta na autenticacao.
 - testar regex na definicao das perguntas. como se faz rawstring em yaml? singlequote? problemas de backslash??? sim... necessário fazer \\ em varios casos, mas não é claro! e.g. \n é convertido em espaço mas \w é convertido em \\ e w.
-- testar envio de parametros para stdin para perguntas tipo generator.
  
 # TODO
  
@@ -26,6 +25,7 @@
  
 # FIXED
  
+- testar envio de parametros para stdin para perguntas tipo generator.
 - mathjax e jquery no login
 - mostrar erro quando nao consegue importar questions files
 - pacotes exactos usados para instalar.
@@ -2,29 +2,29 @@
  
 ### Requirements and instalation
  
-Install:
-
-- python3.4
-- cherrypy3
-- mako
-- yaml
-- markdown
+Install python 3.4 and the following packages from pip:
  
+- CherryPy (3.7.0)
+- Mako (1.0.1)
+- Markdown (2.6.2)
+- PyYAML (3.11)
  
 Before using the program you need to
  
 1. Create the students database
 1. Create questions
 1. Create a test
-1. Configure the server (the default may be enough)
+1. Configure the server (the default should be enough)
  
 ### Create students database
  
 We need a sqlite3 database to store students, passwords, test results, and questions results, etc.
  
-The database can be initialized from a list of students in CSV format using the script
+The database can be initialized from a list of students in CSV format by running in the terminal
  
-    $ ./initdb_from_csv.py list_of_students.csv
+```sh
+./initdb_from_csv.py list_of_students.csv
+```
  
 This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords.
 It also adds a special user number 0. This is the administrator user (Professor).
@@ -36,24 +36,26 @@ The passwords will be defined on the first login.
 Questions are defined in `yaml` files and can reside anywhere in the filesystem.
 Each file contains a list of questions, where each question is a dictionary. Example
  
-    -
-        ref: question-1
-        type: radio
-        text: Select the correct option
-        options:
-            - correct
-            - wrong
-
-    -
-        ref: question-2
-        type: checkbox
-        text: Which ones are correct?
-        options:
-            - correct
-            - correct
-            - wrong
-        correct: [1, 1, -1]
-        hint: There are two correct answers!
+```yaml
+-
+    ref: question-1
+    type: radio
+    text: Select the correct option
+    options:
+        - correct
+        - wrong
+
+-
+    ref: question-2
+    type: checkbox
+    text: Which ones are correct?
+    options:
+        - correct
+        - correct
+        - wrong
+    correct: [1, 1, -1]
+    hint: There are two correct answers!
+```
  
 There are several kinds of questions:
  
@@ -72,84 +74,90 @@ Detailed information on each question type is described later on.
  
 A test is a file in `yaml` format that can reside anywhere on the filesystem. It has the following structure:
  
-    ref: this-is-a-key
-    title: Titulo do teste
-    database: db/mystudents.db
-
-    # Will save the entire test of each student in JSON format.
-    # If tests are to be saved, we must specify the directory.
-    # The directory is created if it doesn't exist already.
-    # The name of the JSON files will include the student number, test
-    # reference key, date and time.
-    save_answers: True
-    answers_dir: ans/asc1_test4
-
-    # Some questions can contain hints, embedded videos, etc
-    show_hints: True
-
-    # Each question has some number of points. Show them normalized to 0-20.
-    show_points: True
-
-    # In train mode, the correction of the test is shown and the test can
-    # be repeated
-    practice_mode: True
-
-    # Show the data structures obtained from the test and the questions
-    debug: False
-
-    # Show the file and ref field of each question
-    show_ref: True
-
-    # -------------------------------------------------------------------------
-    # This are the questions database to be imported.
-    files:
-        - questions/file1.yaml
-        - questions/file2.yaml
-        - questions/file3.yaml
-    # -------------------------------------------------------------------------
-    # This is the actual test configuration. Selection of questions and points
-    # It'a defined as a list of questions. Each question can be a single
-    # question key or a list of keys from which one is chosen at random.
-    # Each question has a default value of 1.0 point, but it can be overridden.
-    # The points defined here do not need to be normalized (it's automatic).
-    questions:
-        -   ref:
-                - first-question-1    # randomly choose one from these 3 questions
-                - first-question-2
-                - first-question-3
-            points: 0.5
-
-        -   ref: second-question    # one question, 1.0 point (unnormalized)
-
-        -   third-question          # "ref:" not needed in simple cases
+```yaml
+ref: this-is-a-key
+title: Titulo do teste
+database: db/mystudents.db
+
+# Will save the entire test of each student in JSON format.
+# If tests are to be saved, we must specify the directory.
+# The directory is created if it doesn't exist already.
+# The name of the JSON files will include the student number, test
+# reference key, date and time.
+save_answers: True
+answers_dir: ans/asc1_test4
+
+# Some questions can contain hints, embedded videos, etc
+show_hints: True
+
+# Each question has some number of points. Show them normalized to 0-20.
+show_points: True
+
+# In train mode, the correction of the test is shown and the test can
+# be repeated
+practice_mode: True
+
+# Show the data structures obtained from the test and the questions
+debug: False
+
+# Show the file and ref field of each question
+show_ref: True
+
+# -------------------------------------------------------------------------
+# This are the questions database to be imported.
+files:
+    - questions/file1.yaml
+    - questions/file2.yaml
+    - questions/file3.yaml
+# -------------------------------------------------------------------------
+# This is the actual test configuration. Selection of questions and points
+# It'a defined as a list of questions. Each question can be a single
+# question key or a list of keys from which one is chosen at random.
+# Each question has a default value of 1.0 point, but it can be overridden.
+# The points defined here do not need to be normalized (it's automatic).
+questions:
+    -   ref:
+            - first-question-1    # randomly choose one from these 3 questions
+            - first-question-2
+            - first-question-3
+        points: 0.5
+
+    -   ref: second-question    # one question, 1.0 point (unnormalized)
+
+    -   third-question          # "ref:" not needed in simple cases
  
 This following one is wrong:
  
-        -   wrong-question          # missing "ref:" key
-            points: 2               
+    -   wrong-question          # missing "ref:" key
+        points: 2               
+```
  
 Some of the options have default values if they are omitted. The defaults are the following:
  
-    ref: filename.yaml
-    title: ''
-    save_answers: False
-    show_hints: False
-    show_points: False
-    practice_mode: False
-    show_ref: False
-    debug: False
-    points: 1.0
-
+```yaml
+ref: filename.yaml
+title: ''
+save_answers: False
+show_hints: False
+show_points: False
+practice_mode: False
+show_ref: False
+debug: False
+points: 1.0
+```
 ### Running an existing test
  
 A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument:
  
-    $ ./serve.py tests_dir/mytest.yaml
+```sh
+$ ./serve.py tests_dir/mytest.yaml
+```
  
 Some defaults can be overriden with command line options. Example
  
-    $ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
-
+```sh
+$ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
+```
 To terminate the test just do `^C` on the keyboard.
  
 ## Questions
@@ -160,29 +168,31 @@ Every question should have a `ref` and a `type`. The other keys depend on the ty
  
 Not a real question. Just text to be shown without expecting an answer.
  
-    -
-        ref: some-key
-        type: information
-        text: Tomorrow will rain.
-
+```yaml
+-
+    ref: some-key
+    type: information
+    text: Tomorrow will rain.
+```
 Correcting an information will always be considered correct, but the grade will be zero because it has 0.0 points by default.
  
 ### Radio
  
 Only one option is correct.
  
-    -
-        ref: some-key
-        type: radio
-        text: The horse is white.      # optional (default: '')
-        options:
-            - The horse is white
-            - The horse is not black
-            - The horse is black
-        correct: 0              # optional (default: 0). Index is 0-based.
-        shuffle: True           # optional (default: True)
-        discount: True          # optional (default: True)
-
+```yaml
+-
+    ref: some-key
+    type: radio
+    text: The horse is white.      # optional (default: '')
+    options:
+        - The horse is white
+        - The horse is not black
+        - The horse is black
+    correct: 0              # optional (default: 0). Index is 0-based.
+    shuffle: True           # optional (default: True)
+    discount: True          # optional (default: True)
+```
 The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`.
  
 Wrong answers discount by default. If there are half-right answers, the discount values are calculated automatically. `discount: False` disables the discount calculation and the values are the ones defined in `correct`.
@@ -191,20 +201,22 @@ Wrong answers discount by default. If there are half-right answers, the discount
  
 There can be several options correct. Each option is like answering an independent question.
  
-    -
-        ref: some-key
-        type: checkbox
-        text: The horse is white.      # optional (default: '')
-        options:
-            - The horse is white
-            - The horse is not black
-            - The horse is black
-        correct: [1,1,-1]              # optional (default: [0,0,0]).
-        shuffle: True                  # optional (default: True)
-        discount: True                 # optional (default: True)
-
+```yaml
+-
+    ref: some-key
+    type: checkbox
+    text: The horse is white.      # optional (default: '')
+    options:
+        - The horse is white
+        - The horse is not black
+        - The horse is black
+    correct: [1,1,-1]              # optional (default: [0,0,0]).
+    shuffle: True                  # optional (default: True)
+    discount: True                 # optional (default: True)
+```
 Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value.
 E.g. consider `correct: [1, 0.5, -1]`, then
+
 - if the first option is marked the value is 1, otherwise if it's unmarked the value is -1.
 - if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5.
 - if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one)
@@ -216,26 +228,29 @@ E.g. consider `correct: [1, 0.5, -1]`, then
 The answer is a line of text.
 The server will check if the answer exactly matches the correct one.
  
-    -
-        ref: some-key
-        type: text
-        text: What's your favorite color?      # optional (default: '')
-        correct: white
-
+```yaml
+-
+    ref: some-key
+    type: text
+    text: What's your favorite color?      # optional (default: '')
+    correct: white
+```
 alternatively, we can give a list of acceptable answers
-
-        correct: ['white', 'blue', 'red']
-
+```yaml
+    correct: ['white', 'blue', 'red']
+```
 ### Regular expression
  
 The answer is a line of text.
 The server will check if the answer matches a regular expression.
  
-    -
-        ref: some-key
-        type: text_regex
-        text: What's your favorite color?      # optional (default: '')
-        correct: '[Ww]hite'
+```yaml
+-
+    ref: some-key
+    type: text_regex
+    text: What's your favorite color?      # optional (default: '')
+    correct: '[Ww]hite'
+```
  
 Careful: yaml does not support raw text. Some characters have to be escaped.
  
@@ -245,23 +260,27 @@ The answer is given in a textarea. The text (usually code) is sent to an externa
 The external program should accept input from stdin, and print to stdout a single number in the interval 0.0 to 1.0 indicating the level of correctness.
 The server will try to convert the printed message to a float, a failure will give 0.0.
  
-    -
-        ref: some-key
-        type: textarea
-        text: write an expression to add x and y.    # optional (default: '')
-        correct: path/to/myscript
+```yaml
+-
+    ref: some-key
+    type: textarea
+    text: write an expression to add x and y.    # optional (default: '')
+    correct: path/to/myscript
+```
  
 An example of a script in python that validades an answer is
  
-    #!/usr/bin/env python3.4
+```python
+#!/usr/bin/env python3.4
  
-    import sys
-    s = sys.stdin.read()
-    if s == 'Alibaba':
-        print(1.0)
-    else:
-        print(0.0)
-    exit(0)
+import sys
+s = sys.stdin.read()
+if s == 'Alibaba':
+    print(1.0)
+else:
+    print(0.0)
+exit(0)
+```
  
 but any script language or executable program can be used for this purpose.
  
@@ -270,25 +289,38 @@ but any script language or executable program can be used for this purpose.
  
 A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!).
  
-    -
-        ref: some-key
-        type: generator
-        script: path/to/generator_script
+```yaml
+-
+    ref: some-key
+    type: generator
+    script: path/to/generator_script
+    # arg: "optional string passed on to stdin of the script"
+```
  
 An example of a question generator is the following
  
-    #!/usr/bin/env python3.4
-    from random import randint
-
-    x = randint(10,20)
-    y = randint(10,20)
-    s = '''
-    ref: addition
-    type: text
-    text: How much is {0} plus {1}?
-    correct: {2}
-    '''.format(x, y, x + y)
-    print(s)
+```python
+#!/usr/bin/env python3.4
+from random import randint
+import sys
+
+# read arguments from stdin and convert to integers
+arg = sys.stdin.read()
+a,b = (int(n) for n in arg.split(','))
+
+# generate question
+x = randint(a, b)
+y = randint(a, b)
+s = '''
+ref: addition
+type: text
+text: How much is {0} plus {1}?
+correct: {2}
+'''.format(x, y, x + y)
+
+# send question to stdout
+print(s)
+```
  
 ## Writing good looking questions
  
@@ -296,13 +328,16 @@ The text of the questions (and options in radio and checkbox type questios) is p
  
 A good way to define multiple lines of text in the questions is to use the bar |. Yaml will use all the text that is indented to the right of that column. Example
  
-        text:   |
-                Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
-                and pretty code in several languages
+```yaml
+    text:   |
+            Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
+            and pretty code in several languages
+
+            ```.C
+            int main(){
+                return 0;
+            }
+            ```
+    # this line stops the text because it is not indented
+```
  
-                ```.C
-                int main(){
-                    return 0;
-                }
-                ```
-        # this line does stops the text because it is not indented
 #!/usr/bin/env python3.4
  
 from random import randint
+import sys
  
+arg = sys.stdin.read()  # read arguments
+
+a,b = (int(n) for n in arg.split(','))
  
 q = '''
 type: checkbox
-text: Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
+text:   |
+        Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
+
+        Os números foram gerados aleatoriamente no intervalo de {0} a {1}.
 options:
-'''
+'''.format(a,b)
  
 correct = []
 for i in range(5):
-    x = randint(11,120)
-    y = randint(11,120)
+    x = randint(a, b)
+    y = randint(a, b)
     q += '- "`{} + {}`"\n'.format(x, y)
     correct.append(1 if x + y > 127 else -1)
  
@@ -51,6 +51,7 @@
     ref: question-whatever
     type: generator
     script: demo/generate-question.py
+    arg: "11,120"
     # the script should print a question in yaml format like the ones above.
     # Print only the dictionary, not the list (hiffen).
 # ---------------------------------------------------------------------------
@@ -114,7 +114,7 @@ def question_generator(q):
     '''Run an external script that will generate a question in yaml format.
     This function will return the yaml converted back to a dict.'''
  
-    q['stdin'] = q.get('stdin', '')
+    q['arg'] = q.get('arg', '')  # send this string to stdin
  
     try:
         p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
@@ -122,7 +122,7 @@ def question_generator(q):
         print('  * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))
  
     try:
-        qyaml = p.communicate(input=q['stdin'].encode('utf-8'), timeout=5)[0].decode('utf-8')
+        qyaml = p.communicate(input=q['arg'].encode('utf-8'), timeout=5)[0].decode('utf-8')
     except subprocess.TimeoutExpired:
         p.kill()
...	...	@@ -7,7 +7,6 @@
7	7	- parece que é preciso criar à mão a pasta para as respostas (ans/...) depois apercebo-me que os caminhos no teste dizem respeito à directoria donde o teste é corrido... as respostas deveriam guardadas no directório dado.
8	8	- se database for mal configurada, é criada uma base de dados vazia e rebenta na autenticacao.
9	9	- testar regex na definicao das perguntas. como se faz rawstring em yaml? singlequote? problemas de backslash??? sim... necessário fazer \\ em varios casos, mas não é claro! e.g. \n é convertido em espaço mas \w é convertido em \\ e w.
10		-- testar envio de parametros para stdin para perguntas tipo generator.
11	10
12	11	# TODO
13	12
...	...	@@ -26,6 +25,7 @@
26	25
27	26	# FIXED
28	27
	28	+- testar envio de parametros para stdin para perguntas tipo generator.
29	29	- mathjax e jquery no login
30	30	- mostrar erro quando nao consegue importar questions files
31	31	- pacotes exactos usados para instalar.
...	...
...	...	@@ -2,29 +2,29 @@
2	2
3	3	### Requirements and instalation
4	4
5		-Install:
6		-
7		-- python3.4
8		-- cherrypy3
9		-- mako
10		-- yaml
11		-- markdown
	5	+Install python 3.4 and the following packages from pip:
12	6
	7	+- CherryPy (3.7.0)
	8	+- Mako (1.0.1)
	9	+- Markdown (2.6.2)
	10	+- PyYAML (3.11)
13	11
14	12	Before using the program you need to
15	13
16	14	1. Create the students database
17	15	1. Create questions
18	16	1. Create a test
19		-1. Configure the server (the default may be enough)
	17	+1. Configure the server (the default should be enough)
20	18
21	19	### Create students database
22	20
23	21	We need a sqlite3 database to store students, passwords, test results, and questions results, etc.
24	22
25		-The database can be initialized from a list of students in CSV format using the script
	23	+The database can be initialized from a list of students in CSV format by running in the terminal
26	24
27		- $ ./initdb_from_csv.py list_of_students.csv
	25	+```sh
	26	+./initdb_from_csv.py list_of_students.csv
	27	+```
28	28
29	29	This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords.
30	30	It also adds a special user number 0. This is the administrator user (Professor).
...	...	@@ -36,24 +36,26 @@ The passwords will be defined on the first login.
36	36	Questions are defined in `yaml` files and can reside anywhere in the filesystem.
37	37	Each file contains a list of questions, where each question is a dictionary. Example
38	38
39		- -
40		- ref: question-1
41		- type: radio
42		- text: Select the correct option
43		- options:
44		- - correct
45		- - wrong
46		-
47		- -
48		- ref: question-2
49		- type: checkbox
50		- text: Which ones are correct?
51		- options:
52		- - correct
53		- - correct
54		- - wrong
55		- correct: [1, 1, -1]
56		- hint: There are two correct answers!
	39	+```yaml
	40	+-
	41	+ ref: question-1
	42	+ type: radio
	43	+ text: Select the correct option
	44	+ options:
	45	+ - correct
	46	+ - wrong
	47	+
	48	+-
	49	+ ref: question-2
	50	+ type: checkbox
	51	+ text: Which ones are correct?
	52	+ options:
	53	+ - correct
	54	+ - correct
	55	+ - wrong
	56	+ correct: [1, 1, -1]
	57	+ hint: There are two correct answers!
	58	+```
57	59
58	60	There are several kinds of questions:
59	61
...	...	@@ -72,84 +74,90 @@ Detailed information on each question type is described later on.
72	74
73	75	A test is a file in `yaml` format that can reside anywhere on the filesystem. It has the following structure:
74	76
75		- ref: this-is-a-key
76		- title: Titulo do teste
77		- database: db/mystudents.db
78		-
79		- # Will save the entire test of each student in JSON format.
80		- # If tests are to be saved, we must specify the directory.
81		- # The directory is created if it doesn't exist already.
82		- # The name of the JSON files will include the student number, test
83		- # reference key, date and time.
84		- save_answers: True
85		- answers_dir: ans/asc1_test4
86		-
87		- # Some questions can contain hints, embedded videos, etc
88		- show_hints: True
89		-
90		- # Each question has some number of points. Show them normalized to 0-20.
91		- show_points: True
92		-
93		- # In train mode, the correction of the test is shown and the test can
94		- # be repeated
95		- practice_mode: True
96		-
97		- # Show the data structures obtained from the test and the questions
98		- debug: False
99		-
100		- # Show the file and ref field of each question
101		- show_ref: True
102		-
103		- # -------------------------------------------------------------------------
104		- # This are the questions database to be imported.
105		- files:
106		- - questions/file1.yaml
107		- - questions/file2.yaml
108		- - questions/file3.yaml
109		- # -------------------------------------------------------------------------
110		- # This is the actual test configuration. Selection of questions and points
111		- # It'a defined as a list of questions. Each question can be a single
112		- # question key or a list of keys from which one is chosen at random.
113		- # Each question has a default value of 1.0 point, but it can be overridden.
114		- # The points defined here do not need to be normalized (it's automatic).
115		- questions:
116		- - ref:
117		- - first-question-1 # randomly choose one from these 3 questions
118		- - first-question-2
119		- - first-question-3
120		- points: 0.5
121		-
122		- - ref: second-question # one question, 1.0 point (unnormalized)
123		-
124		- - third-question # "ref:" not needed in simple cases
	77	+```yaml
	78	+ref: this-is-a-key
	79	+title: Titulo do teste
	80	+database: db/mystudents.db
	81	+
	82	+# Will save the entire test of each student in JSON format.
	83	+# If tests are to be saved, we must specify the directory.
	84	+# The directory is created if it doesn't exist already.
	85	+# The name of the JSON files will include the student number, test
	86	+# reference key, date and time.
	87	+save_answers: True
	88	+answers_dir: ans/asc1_test4
	89	+
	90	+# Some questions can contain hints, embedded videos, etc
	91	+show_hints: True
	92	+
	93	+# Each question has some number of points. Show them normalized to 0-20.
	94	+show_points: True
	95	+
	96	+# In train mode, the correction of the test is shown and the test can
	97	+# be repeated
	98	+practice_mode: True
	99	+
	100	+# Show the data structures obtained from the test and the questions
	101	+debug: False
	102	+
	103	+# Show the file and ref field of each question
	104	+show_ref: True
	105	+
	106	+# -------------------------------------------------------------------------
	107	+# This are the questions database to be imported.
	108	+files:
	109	+ - questions/file1.yaml
	110	+ - questions/file2.yaml
	111	+ - questions/file3.yaml
	112	+# -------------------------------------------------------------------------
	113	+# This is the actual test configuration. Selection of questions and points
	114	+# It'a defined as a list of questions. Each question can be a single
	115	+# question key or a list of keys from which one is chosen at random.
	116	+# Each question has a default value of 1.0 point, but it can be overridden.
	117	+# The points defined here do not need to be normalized (it's automatic).
	118	+questions:
	119	+ - ref:
	120	+ - first-question-1 # randomly choose one from these 3 questions
	121	+ - first-question-2
	122	+ - first-question-3
	123	+ points: 0.5
	124	+
	125	+ - ref: second-question # one question, 1.0 point (unnormalized)
	126	+
	127	+ - third-question # "ref:" not needed in simple cases
125	128
126	129	This following one is wrong:
127	130
128		- - wrong-question # missing "ref:" key
129		- points: 2
	131	+ - wrong-question # missing "ref:" key
	132	+ points: 2
	133	+```
130	134
131	135	Some of the options have default values if they are omitted. The defaults are the following:
132	136
133		- ref: filename.yaml
134		- title: ''
135		- save_answers: False
136		- show_hints: False
137		- show_points: False
138		- practice_mode: False
139		- show_ref: False
140		- debug: False
141		- points: 1.0
142		-
	137	+```yaml
	138	+ref: filename.yaml
	139	+title: ''
	140	+save_answers: False
	141	+show_hints: False
	142	+show_points: False
	143	+practice_mode: False
	144	+show_ref: False
	145	+debug: False
	146	+points: 1.0
	147	+```
143	148	### Running an existing test
144	149
145	150	A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument:
146	151
147		- $ ./serve.py tests_dir/mytest.yaml
	152	+```sh
	153	+$ ./serve.py tests_dir/mytest.yaml
	154	+```
148	155
149	156	Some defaults can be overriden with command line options. Example
150	157
151		- $ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
152		-
	158	+```sh
	159	+$ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
	160	+```
153	161	To terminate the test just do `^C` on the keyboard.
154	162
155	163	## Questions
...	...	@@ -160,29 +168,31 @@ Every question should have a `ref` and a `type`. The other keys depend on the ty
160	168
161	169	Not a real question. Just text to be shown without expecting an answer.
162	170
163		- -
164		- ref: some-key
165		- type: information
166		- text: Tomorrow will rain.
167		-
	171	+```yaml
	172	+-
	173	+ ref: some-key
	174	+ type: information
	175	+ text: Tomorrow will rain.
	176	+```
168	177	Correcting an information will always be considered correct, but the grade will be zero because it has 0.0 points by default.
169	178
170	179	### Radio
171	180
172	181	Only one option is correct.
173	182
174		- -
175		- ref: some-key
176		- type: radio
177		- text: The horse is white. # optional (default: '')
178		- options:
179		- - The horse is white
180		- - The horse is not black
181		- - The horse is black
182		- correct: 0 # optional (default: 0). Index is 0-based.
183		- shuffle: True # optional (default: True)
184		- discount: True # optional (default: True)
185		-
	183	+```yaml
	184	+-
	185	+ ref: some-key
	186	+ type: radio
	187	+ text: The horse is white. # optional (default: '')
	188	+ options:
	189	+ - The horse is white
	190	+ - The horse is not black
	191	+ - The horse is black
	192	+ correct: 0 # optional (default: 0). Index is 0-based.
	193	+ shuffle: True # optional (default: True)
	194	+ discount: True # optional (default: True)
	195	+```
186	196	The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`.
187	197
188	198	Wrong answers discount by default. If there are half-right answers, the discount values are calculated automatically. `discount: False` disables the discount calculation and the values are the ones defined in `correct`.
...	...	@@ -191,20 +201,22 @@ Wrong answers discount by default. If there are half-right answers, the discount
191	201
192	202	There can be several options correct. Each option is like answering an independent question.
193	203
194		- -
195		- ref: some-key
196		- type: checkbox
197		- text: The horse is white. # optional (default: '')
198		- options:
199		- - The horse is white
200		- - The horse is not black
201		- - The horse is black
202		- correct: [1,1,-1] # optional (default: [0,0,0]).
203		- shuffle: True # optional (default: True)
204		- discount: True # optional (default: True)
205		-
	204	+```yaml
	205	+-
	206	+ ref: some-key
	207	+ type: checkbox
	208	+ text: The horse is white. # optional (default: '')
	209	+ options:
	210	+ - The horse is white
	211	+ - The horse is not black
	212	+ - The horse is black
	213	+ correct: [1,1,-1] # optional (default: [0,0,0]).
	214	+ shuffle: True # optional (default: True)
	215	+ discount: True # optional (default: True)
	216	+```
206	217	Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value.
207	218	E.g. consider `correct: [1, 0.5, -1]`, then
	219	+
208	220	- if the first option is marked the value is 1, otherwise if it's unmarked the value is -1.
209	221	- if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5.
210	222	- if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one)
...	...	@@ -216,26 +228,29 @@ E.g. consider `correct: [1, 0.5, -1]`, then
216	228	The answer is a line of text.
217	229	The server will check if the answer exactly matches the correct one.
218	230
219		- -
220		- ref: some-key
221		- type: text
222		- text: What's your favorite color? # optional (default: '')
223		- correct: white
224		-
	231	+```yaml
	232	+-
	233	+ ref: some-key
	234	+ type: text
	235	+ text: What's your favorite color? # optional (default: '')
	236	+ correct: white
	237	+```
225	238	alternatively, we can give a list of acceptable answers
226		-
227		- correct: ['white', 'blue', 'red']
228		-
	239	+```yaml
	240	+ correct: ['white', 'blue', 'red']
	241	+```
229	242	### Regular expression
230	243
231	244	The answer is a line of text.
232	245	The server will check if the answer matches a regular expression.
233	246
234		- -
235		- ref: some-key
236		- type: text_regex
237		- text: What's your favorite color? # optional (default: '')
238		- correct: '[Ww]hite'
	247	+```yaml
	248	+-
	249	+ ref: some-key
	250	+ type: text_regex
	251	+ text: What's your favorite color? # optional (default: '')
	252	+ correct: '[Ww]hite'
	253	+```
239	254
240	255	Careful: yaml does not support raw text. Some characters have to be escaped.
241	256
...	...	@@ -245,23 +260,27 @@ The answer is given in a textarea. The text (usually code) is sent to an externa
245	260	The external program should accept input from stdin, and print to stdout a single number in the interval 0.0 to 1.0 indicating the level of correctness.
246	261	The server will try to convert the printed message to a float, a failure will give 0.0.
247	262
248		- -
249		- ref: some-key
250		- type: textarea
251		- text: write an expression to add x and y. # optional (default: '')
252		- correct: path/to/myscript
	263	+```yaml
	264	+-
	265	+ ref: some-key
	266	+ type: textarea
	267	+ text: write an expression to add x and y. # optional (default: '')
	268	+ correct: path/to/myscript
	269	+```
253	270
254	271	An example of a script in python that validades an answer is
255	272
256		- #!/usr/bin/env python3.4
	273	+```python
	274	+#!/usr/bin/env python3.4
257	275
258		- import sys
259		- s = sys.stdin.read()
260		- if s == 'Alibaba':
261		- print(1.0)
262		- else:
263		- print(0.0)
264		- exit(0)
	276	+import sys
	277	+s = sys.stdin.read()
	278	+if s == 'Alibaba':
	279	+ print(1.0)
	280	+else:
	281	+ print(0.0)
	282	+exit(0)
	283	+```
265	284
266	285	but any script language or executable program can be used for this purpose.
267	286
...	...	@@ -270,25 +289,38 @@ but any script language or executable program can be used for this purpose.
270	289
271	290	A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!).
272	291
273		- -
274		- ref: some-key
275		- type: generator
276		- script: path/to/generator_script
	292	+```yaml
	293	+-
	294	+ ref: some-key
	295	+ type: generator
	296	+ script: path/to/generator_script
	297	+ # arg: "optional string passed on to stdin of the script"
	298	+```
277	299
278	300	An example of a question generator is the following
279	301
280		- #!/usr/bin/env python3.4
281		- from random import randint
282		-
283		- x = randint(10,20)
284		- y = randint(10,20)
285		- s = '''
286		- ref: addition
287		- type: text
288		- text: How much is {0} plus {1}?
289		- correct: {2}
290		- '''.format(x, y, x + y)
291		- print(s)
	302	+```python
	303	+#!/usr/bin/env python3.4
	304	+from random import randint
	305	+import sys
	306	+
	307	+# read arguments from stdin and convert to integers
	308	+arg = sys.stdin.read()
	309	+a,b = (int(n) for n in arg.split(','))
	310	+
	311	+# generate question
	312	+x = randint(a, b)
	313	+y = randint(a, b)
	314	+s = '''
	315	+ref: addition
	316	+type: text
	317	+text: How much is {0} plus {1}?
	318	+correct: {2}
	319	+'''.format(x, y, x + y)
	320	+
	321	+# send question to stdout
	322	+print(s)
	323	+```
292	324
293	325	## Writing good looking questions
294	326
...	...	@@ -296,13 +328,16 @@ The text of the questions (and options in radio and checkbox type questios) is p
296	328
297	329	A good way to define multiple lines of text in the questions is to use the bar \|. Yaml will use all the text that is indented to the right of that column. Example
298	330
299		- text: \|
300		- Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
301		- and pretty code in several languages
	331	+```yaml
	332	+ text: \|
	333	+ Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
	334	+ and pretty code in several languages
	335	+
	336	+ ```.C
	337	+ int main(){
	338	+ return 0;
	339	+ }
	340	+ ```
	341	+ # this line stops the text because it is not indented
	342	+```
302	343
303		- ```.C
304		- int main(){
305		- return 0;
306		- }
307		- ```
308		- # this line does stops the text because it is not indented
...	...
1	1	#!/usr/bin/env python3.4
2	2
3	3	from random import randint
	4	+import sys
4	5
	6	+arg = sys.stdin.read() # read arguments
	7	+
	8	+a,b = (int(n) for n in arg.split(','))
5	9
6	10	q = '''
7	11	type: checkbox
8		-text: Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
	12	+text: \|
	13	+ Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
	14	+
	15	+ Os números foram gerados aleatoriamente no intervalo de {0} a {1}.
9	16	options:
10		-'''
	17	+'''.format(a,b)
11	18
12	19	correct = []
13	20	for i in range(5):
14		- x = randint(11,120)
15		- y = randint(11,120)
	21	+ x = randint(a, b)
	22	+ y = randint(a, b)
16	23	q += '- "`{} + {}`"\n'.format(x, y)
17	24	correct.append(1 if x + y > 127 else -1)
18	25
...	...
...	...	@@ -51,6 +51,7 @@
51	51	ref: question-whatever
52	52	type: generator
53	53	script: demo/generate-question.py
	54	+ arg: "11,120"
54	55	# the script should print a question in yaml format like the ones above.
55	56	# Print only the dictionary, not the list (hiffen).
56	57	# ---------------------------------------------------------------------------
...	...
...	...	@@ -114,7 +114,7 @@ def question_generator(q):
114	114	'''Run an external script that will generate a question in yaml format.
115	115	This function will return the yaml converted back to a dict.'''
116	116
117		- q['stdin'] = q.get('stdin', '')
	117	+ q['arg'] = q.get('arg', '') # send this string to stdin
118	118
119	119	try:
120	120	p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
...	...	@@ -122,7 +122,7 @@ def question_generator(q):
122	122	print(' * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))
123	123
124	124	try:
125		- qyaml = p.communicate(input=q['stdin'].encode('utf-8'), timeout=5)[0].decode('utf-8')
	125	+ qyaml = p.communicate(input=q['arg'].encode('utf-8'), timeout=5)[0].decode('utf-8')
126	126	except subprocess.TimeoutExpired:
127	127	p.kill()
128	128
...	...