- replaced key 'stdin' by 'arg' in questions of type generator

- updated manual and demo

- replaced key 'stdin' by 'arg' in questions of type generator
- updated manual and demo
Miguel Barao
1 parent 9bace4f6
Showing 5 changed files with 224 additions and 181 deletions Show diff stats
BUGS.md
MANUAL.md
demo/generate-question.py
demo/questions.yaml
questions.py
@@ -7,7 +7,6 @@
 - parece que é preciso criar à mão a pasta para as respostas (ans/...) depois apercebo-me que os caminhos no teste dizem respeito à directoria donde o teste é corrido... as respostas deveriam guardadas no directório dado.
 - se database for mal configurada, é criada uma base de dados vazia e rebenta na autenticacao.
 - testar regex na definicao das perguntas. como se faz rawstring em yaml? singlequote? problemas de backslash??? sim... necessário fazer \\ em varios casos, mas não é claro! e.g. \n é convertido em espaço mas \w é convertido em \\ e w.
-- testar envio de parametros para stdin para perguntas tipo generator.
 # TODO
@@ -26,6 +25,7 @@
 # FIXED
+- testar envio de parametros para stdin para perguntas tipo generator.
 - mathjax e jquery no login
 - mostrar erro quando nao consegue importar questions files
 - pacotes exactos usados para instalar.
@@ -2,29 +2,29 @@
 ### Requirements and instalation
-Install:
-
-- python3.4
-- cherrypy3
-- mako
-- yaml
-- markdown
+Install python 3.4 and the following packages from pip:
+- CherryPy (3.7.0)
+- Mako (1.0.1)
+- Markdown (2.6.2)
+- PyYAML (3.11)
 Before using the program you need to
 1. Create the students database
 1. Create questions
 1. Create a test
-1. Configure the server (the default may be enough)
+1. Configure the server (the default should be enough)
 ### Create students database
 We need a sqlite3 database to store students, passwords, test results, and questions results, etc.
-The database can be initialized from a list of students in CSV format using the script
+The database can be initialized from a list of students in CSV format by running in the terminal
-    $ ./initdb_from_csv.py list_of_students.csv
+```sh
+./initdb_from_csv.py list_of_students.csv
+```
 This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords.
 It also adds a special user number 0. This is the administrator user (Professor).
@@ -36,24 +36,26 @@ The passwords will be defined on the first login.
 Questions are defined in `yaml` files and can reside anywhere in the filesystem.
 Each file contains a list of questions, where each question is a dictionary. Example
-    -
-        ref: question-1
-        type: radio
-        text: Select the correct option
-        options:
-            - correct
-            - wrong
-
-    -
-        ref: question-2
-        type: checkbox
-        text: Which ones are correct?
-        options:
-            - correct
-            - correct
-            - wrong
-        correct: [1, 1, -1]
-        hint: There are two correct answers!
+```yaml
+-
+    ref: question-1
+    type: radio
+    text: Select the correct option
+    options:
+        - correct
+        - wrong
+
+-
+    ref: question-2
+    type: checkbox
+    text: Which ones are correct?
+    options:
+        - correct
+        - correct
+        - wrong
+    correct: [1, 1, -1]
+    hint: There are two correct answers!
+```
 There are several kinds of questions:
@@ -72,84 +74,90 @@ Detailed information on each question type is described later on.
 A test is a file in `yaml` format that can reside anywhere on the filesystem. It has the following structure:
-    ref: this-is-a-key
-    title: Titulo do teste
-    database: db/mystudents.db
-
-    # Will save the entire test of each student in JSON format.
-    # If tests are to be saved, we must specify the directory.
-    # The directory is created if it doesn't exist already.
-    # The name of the JSON files will include the student number, test
-    # reference key, date and time.
-    save_answers: True
-    answers_dir: ans/asc1_test4
-
-    # Some questions can contain hints, embedded videos, etc
-    show_hints: True
-
-    # Each question has some number of points. Show them normalized to 0-20.
-    show_points: True
-
-    # In train mode, the correction of the test is shown and the test can
-    # be repeated
-    practice_mode: True
-
-    # Show the data structures obtained from the test and the questions
-    debug: False
-
-    # Show the file and ref field of each question
-    show_ref: True
-
-    # -------------------------------------------------------------------------
-    # This are the questions database to be imported.
-    files:
-        - questions/file1.yaml
-        - questions/file2.yaml
-        - questions/file3.yaml
-    # -------------------------------------------------------------------------
-    # This is the actual test configuration. Selection of questions and points
-    # It'a defined as a list of questions. Each question can be a single
-    # question key or a list of keys from which one is chosen at random.
-    # Each question has a default value of 1.0 point, but it can be overridden.
-    # The points defined here do not need to be normalized (it's automatic).
-    questions:
-        -   ref:
-                - first-question-1    # randomly choose one from these 3 questions
-                - first-question-2
-                - first-question-3
-            points: 0.5
-
-        -   ref: second-question    # one question, 1.0 point (unnormalized)
-
-        -   third-question          # "ref:" not needed in simple cases
+```yaml
+ref: this-is-a-key
+title: Titulo do teste
+database: db/mystudents.db
+
+# Will save the entire test of each student in JSON format.
+# If tests are to be saved, we must specify the directory.
+# The directory is created if it doesn't exist already.
+# The name of the JSON files will include the student number, test
+# reference key, date and time.
+save_answers: True
+answers_dir: ans/asc1_test4
+
+# Some questions can contain hints, embedded videos, etc
+show_hints: True
+
+# Each question has some number of points. Show them normalized to 0-20.
+show_points: True
+
+# In train mode, the correction of the test is shown and the test can
+# be repeated
+practice_mode: True
+
+# Show the data structures obtained from the test and the questions
+debug: False
+
+# Show the file and ref field of each question
+show_ref: True
+
+# -------------------------------------------------------------------------
+# This are the questions database to be imported.
+files:
+    - questions/file1.yaml
+    - questions/file2.yaml
+    - questions/file3.yaml
+# -------------------------------------------------------------------------
+# This is the actual test configuration. Selection of questions and points
+# It'a defined as a list of questions. Each question can be a single
+# question key or a list of keys from which one is chosen at random.
+# Each question has a default value of 1.0 point, but it can be overridden.
+# The points defined here do not need to be normalized (it's automatic).
+questions:
+    -   ref:
+            - first-question-1    # randomly choose one from these 3 questions
+            - first-question-2
+            - first-question-3
+        points: 0.5
+
+    -   ref: second-question    # one question, 1.0 point (unnormalized)
+
+    -   third-question          # "ref:" not needed in simple cases
 This following one is wrong:
-        -   wrong-question          # missing "ref:" key
-            points: 2               
+    -   wrong-question          # missing "ref:" key
+        points: 2               
+```
 Some of the options have default values if they are omitted. The defaults are the following:
-    ref: filename.yaml
-    title: ''
-    save_answers: False
-    show_hints: False
-    show_points: False
-    practice_mode: False
-    show_ref: False
-    debug: False
-    points: 1.0
-
+```yaml
+ref: filename.yaml
+title: ''
+save_answers: False
+show_hints: False
+show_points: False
+practice_mode: False
+show_ref: False
+debug: False
+points: 1.0
+```
 ### Running an existing test
 A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument:
-    $ ./serve.py tests_dir/mytest.yaml
+```sh
+$ ./serve.py tests_dir/mytest.yaml
+```
 Some defaults can be overriden with command line options. Example
-    $ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
-
+```sh
+$ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers
+```
 To terminate the test just do `^C` on the keyboard.
 ## Questions
@@ -160,29 +168,31 @@ Every question should have a `ref` and a `type`. The other keys depend on the ty
 Not a real question. Just text to be shown without expecting an answer.
-    -
-        ref: some-key
-        type: information
-        text: Tomorrow will rain.
-
+```yaml
+-
+    ref: some-key
+    type: information
+    text: Tomorrow will rain.
+```
 Correcting an information will always be considered correct, but the grade will be zero because it has 0.0 points by default.
 ### Radio
 Only one option is correct.
-    -
-        ref: some-key
-        type: radio
-        text: The horse is white.      # optional (default: '')
-        options:
-            - The horse is white
-            - The horse is not black
-            - The horse is black
-        correct: 0              # optional (default: 0). Index is 0-based.
-        shuffle: True           # optional (default: True)
-        discount: True          # optional (default: True)
-
+```yaml
+-
+    ref: some-key
+    type: radio
+    text: The horse is white.      # optional (default: '')
+    options:
+        - The horse is white
+        - The horse is not black
+        - The horse is black
+    correct: 0              # optional (default: 0). Index is 0-based.
+    shuffle: True           # optional (default: True)
+    discount: True          # optional (default: True)
+```
 The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`.
 Wrong answers discount by default. If there are half-right answers, the discount values are calculated automatically. `discount: False` disables the discount calculation and the values are the ones defined in `correct`.
@@ -191,20 +201,22 @@ Wrong answers discount by default. If there are half-right answers, the discount
 There can be several options correct. Each option is like answering an independent question.
-    -
-        ref: some-key
-        type: checkbox
-        text: The horse is white.      # optional (default: '')
-        options:
-            - The horse is white
-            - The horse is not black
-            - The horse is black
-        correct: [1,1,-1]              # optional (default: [0,0,0]).
-        shuffle: True                  # optional (default: True)
-        discount: True                 # optional (default: True)
-
+```yaml
+-
+    ref: some-key
+    type: checkbox
+    text: The horse is white.      # optional (default: '')
+    options:
+        - The horse is white
+        - The horse is not black
+        - The horse is black
+    correct: [1,1,-1]              # optional (default: [0,0,0]).
+    shuffle: True                  # optional (default: True)
+    discount: True                 # optional (default: True)
+```
 Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value.
 E.g. consider `correct: [1, 0.5, -1]`, then
+
 - if the first option is marked the value is 1, otherwise if it's unmarked the value is -1.
 - if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5.
 - if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one)
@@ -216,26 +228,29 @@ E.g. consider `correct: [1, 0.5, -1]`, then
 The answer is a line of text.
 The server will check if the answer exactly matches the correct one.
-    -
-        ref: some-key
-        type: text
-        text: What's your favorite color?      # optional (default: '')
-        correct: white
-
+```yaml
+-
+    ref: some-key
+    type: text
+    text: What's your favorite color?      # optional (default: '')
+    correct: white
+```
 alternatively, we can give a list of acceptable answers
-
-        correct: ['white', 'blue', 'red']
-
+```yaml
+    correct: ['white', 'blue', 'red']
+```
 ### Regular expression
 The answer is a line of text.
 The server will check if the answer matches a regular expression.
-    -
-        ref: some-key
-        type: text_regex
-        text: What's your favorite color?      # optional (default: '')
-        correct: '[Ww]hite'
+```yaml
+-
+    ref: some-key
+    type: text_regex
+    text: What's your favorite color?      # optional (default: '')
+    correct: '[Ww]hite'
+```
 Careful: yaml does not support raw text. Some characters have to be escaped.
@@ -245,23 +260,27 @@ The answer is given in a textarea. The text (usually code) is sent to an externa
 The external program should accept input from stdin, and print to stdout a single number in the interval 0.0 to 1.0 indicating the level of correctness.
 The server will try to convert the printed message to a float, a failure will give 0.0.
-    -
-        ref: some-key
-        type: textarea
-        text: write an expression to add x and y.    # optional (default: '')
-        correct: path/to/myscript
+```yaml
+-
+    ref: some-key
+    type: textarea
+    text: write an expression to add x and y.    # optional (default: '')
+    correct: path/to/myscript
+```
 An example of a script in python that validades an answer is
-    #!/usr/bin/env python3.4
+```python
+#!/usr/bin/env python3.4
-    import sys
-    s = sys.stdin.read()
-    if s == 'Alibaba':
-        print(1.0)
-    else:
-        print(0.0)
-    exit(0)
+import sys
+s = sys.stdin.read()
+if s == 'Alibaba':
+    print(1.0)
+else:
+    print(0.0)
+exit(0)
+```
 but any script language or executable program can be used for this purpose.
@@ -270,25 +289,38 @@ but any script language or executable program can be used for this purpose.
 A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!).
-    -
-        ref: some-key
-        type: generator
-        script: path/to/generator_script
+```yaml
+-
+    ref: some-key
+    type: generator
+    script: path/to/generator_script
+    # arg: "optional string passed on to stdin of the script"
+```
 An example of a question generator is the following
-    #!/usr/bin/env python3.4
-    from random import randint
-
-    x = randint(10,20)
-    y = randint(10,20)
-    s = '''
-    ref: addition
-    type: text
-    text: How much is {0} plus {1}?
-    correct: {2}
-    '''.format(x, y, x + y)
-    print(s)
+```python
+#!/usr/bin/env python3.4
+from random import randint
+import sys
+
+# read arguments from stdin and convert to integers
+arg = sys.stdin.read()
+a,b = (int(n) for n in arg.split(','))
+
+# generate question
+x = randint(a, b)
+y = randint(a, b)
+s = '''
+ref: addition
+type: text
+text: How much is {0} plus {1}?
+correct: {2}
+'''.format(x, y, x + y)
+
+# send question to stdout
+print(s)
+```
 ## Writing good looking questions
@@ -296,13 +328,16 @@ The text of the questions (and options in radio and checkbox type questios) is p
 A good way to define multiple lines of text in the questions is to use the bar |. Yaml will use all the text that is indented to the right of that column. Example
-        text:   |
-                Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
-                and pretty code in several languages
+```yaml
+    text:   |
+            Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
+            and pretty code in several languages
+
+            ```.C
+            int main(){
+                return 0;
+            }
+            ```
+    # this line stops the text because it is not indented
+```
-                ```.C
-                int main(){
-                    return 0;
-                }
-                ```
-        # this line does stops the text because it is not indented
 #!/usr/bin/env python3.4
 from random import randint
+import sys
+arg = sys.stdin.read()  # read arguments
+
+a,b = (int(n) for n in arg.split(','))
 q = '''
 type: checkbox
-text: Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
+text:   |
+        Indique quais das seguintes adições resultam em overflow quando se considera a adição de números com sinal (complemento para 2) em registos de 8 bits.
+
+        Os números foram gerados aleatoriamente no intervalo de {0} a {1}.
 options:
-'''
+'''.format(a,b)
 correct = []
 for i in range(5):
-    x = randint(11,120)
-    y = randint(11,120)
+    x = randint(a, b)
+    y = randint(a, b)
     q += '- "`{} + {}`"\n'.format(x, y)
     correct.append(1 if x + y > 127 else -1)
@@ -51,6 +51,7 @@
     ref: question-whatever
     type: generator
     script: demo/generate-question.py
+    arg: "11,120"
     # the script should print a question in yaml format like the ones above.
     # Print only the dictionary, not the list (hiffen).
 # ---------------------------------------------------------------------------
@@ -114,7 +114,7 @@ def question_generator(q):
     '''Run an external script that will generate a question in yaml format.
     This function will return the yaml converted back to a dict.'''
-    q['stdin'] = q.get('stdin', '')
+    q['arg'] = q.get('arg', '')  # send this string to stdin
     try:
         p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
@@ -122,7 +122,7 @@ def question_generator(q):
         print('  * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))
     try:
-        qyaml = p.communicate(input=q['stdin'].encode('utf-8'), timeout=5)[0].decode('utf-8')
+        qyaml = p.communicate(input=q['arg'].encode('utf-8'), timeout=5)[0].decode('utf-8')
     except subprocess.TimeoutExpired:
         p.kill()
	@@ -114,7 +114,7 @@ def question_generator(q):		@@ -114,7 +114,7 @@ def question_generator(q):
114	'''Run an external script that will generate a question in yaml format.	114	'''Run an external script that will generate a question in yaml format.
115	This function will return the yaml converted back to a dict.'''	115	This function will return the yaml converted back to a dict.'''
116		116
117	- q['stdin'] = q.get('stdin', '')	117	+ q['arg'] = q.get('arg', '') # send this string to stdin
118		118
119	try:	119	try:
120	p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)	120	p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
	@@ -122,7 +122,7 @@ def question_generator(q):		@@ -122,7 +122,7 @@ def question_generator(q):
122	print(' * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))	122	print(' * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))
123		123
124	try:	124	try:
125	- qyaml = p.communicate(input=q['stdin'].encode('utf-8'), timeout=5)[0].decode('utf-8')	125	+ qyaml = p.communicate(input=q['arg'].encode('utf-8'), timeout=5)[0].decode('utf-8')
126	except subprocess.TimeoutExpired:	126	except subprocess.TimeoutExpired:
127	p.kill()	127	p.kill()
128		128