- added try/except with more informative messages.

- replaced "practice_mode" by "practice".

- added try/except with more informative messages.
- replaced "practice_mode" by "practice".
Miguel Barao
1 parent d69df6c5
Showing 7 changed files with 109 additions and 50 deletions Show diff stats
BUGS.md
MANUAL.md
database.py
questions.py
serve.py
templates/test.html
test.py
@@ -11,6 +11,7 @@
 # TODO
+- Quando apresenta o teste, preencher com os valores definidos em answer (permite que professor dê informação à partida, e no modo practice fiquem com o preenchido anteriormente)
 - testar envio de parametros para stdin para perguntas tipo generator
 - permitir enviar varios testes, aluno escolhe qual o teste que quer fazer.
 - criar script json2md.py ou outra forma de gerar um teste ja realizado
-# Perguntas
+# Perguntations
-## Quick How to
+Before using the program you need to
+
+1. Create the students database
+1. Create questions
+1. Create a test
+1. Configure the server (the default may be enough)
 ### Create students database
@@ -13,6 +18,8 @@ The database can be initialized from a list of students in CSV format using the 
 This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords.
 It also adds a special user number 0. This is the administrator user (Professor).
+The passwords will be defined on the first login.
+
 ### Create new questions
 Questions are defined in `yaml` files and can reside anywhere in the filesystem.
@@ -25,7 +32,7 @@ Each file contains a list of questions, where each question is a dictionary. Exa
         options:
             - correct
             - wrong
-    
+
     -
         ref: question-2
         type: checkbox
@@ -57,28 +64,28 @@ A test is a file in `yaml` format that can reside anywhere on the filesystem. It
     ref: this-is-a-key
     title: Titulo do teste
     database: db/mystudents.db
-    
+
     # Will save the entire test of each student in JSON format.
     # If tests are to be saved, we must specify the directory.
     # The directory is created if it doesn't exist already.
-    # The name of the JSON files will include the student number, test 
+    # The name of the JSON files will include the student number, test
     # reference key, date and time.
-    save_answers: True                 
+    save_answers: True
     answers_dir: ans/asc1_test4
-    
+
     # Some questions can contain hints, embedded videos, etc
     show_hints: True
-    
+
     # Each question has some number of points. Show them normalized to 0-20.
     show_points: True
-    
+
     # In train mode, the correction of the test is shown and the test can
     # be repeated
     practice_mode: True
-    
+
     # Show the data structures obtained from the test and the questions
     debug: True
-    
+
     # -------------------------------------------------------------------------
     # This are the questions database to be imported.
     files:
@@ -87,7 +94,7 @@ A test is a file in `yaml` format that can reside anywhere on the filesystem. It
         - questions/file3.yaml
     # -------------------------------------------------------------------------
     # This is the actual test configuration. Selection of questions and points
-    # It'a defined as a list of questions. Each question can be a single 
+    # It'a defined as a list of questions. Each question can be a single
     # question key or a list of keys from which one is chosen at random.
     # Each question has a default value of 1.0 point, but it can be overridden.
     # The points defined here do not need to be normalized (it's automatic).
@@ -97,11 +104,11 @@ A test is a file in `yaml` format that can reside anywhere on the filesystem. It
             - first-question-2
             - first-question-3
           points: 0.5
-    
+
         - ref: second-question    # just one question, 1.0 point (unnormalized)
-    
+
         - third-question          # "ref:" not needed in simple cases
-    
+
         - wrong-question          # ref: missing because we also have
           points: 2               # points:
@@ -119,7 +126,7 @@ Some of the options have default values if they are omitted. The defaults are th
 ### Running an existing test
 A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument:
-    
+
     $ ./serve.py tests_dir/mytest.yaml
 Some defaults can be overriden with command line options. Example
@@ -145,7 +152,7 @@ Correcting an information will always be considered correct, but the grade will 
 ### Radio
-Only one option is correct. 
+Only one option is correct.
     -
         ref: some-key
@@ -153,11 +160,11 @@ Only one option is correct.
         text: The horse is white.      # optional (default: '')
         options:
             - The horse is white
-            - The horse is not black 
+            - The horse is not black
             - The horse is black
         correct: 0              # optional (default: 0). Index is 0-based.
         shuffle: True           # optional (default: True)
-        discount: True          # optional (default: True) 
+        discount: True          # optional (default: True)
 The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`.
@@ -173,14 +180,14 @@ There can be several options correct. Each option is like answering an independe
         text: The horse is white.      # optional (default: '')
         options:
             - The horse is white
-            - The horse is not black 
+            - The horse is not black
             - The horse is black
         correct: [1,1,-1]              # optional (default: [0,0,0]).
         shuffle: True                  # optional (default: True)
-        discount: True                 # optional (default: True) 
+        discount: True                 # optional (default: True)
 Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value.
-E.g. consider `correct: [1, 0.5, -1]`, then 
+E.g. consider `correct: [1, 0.5, -1]`, then
 - if the first option is marked the value is 1, otherwise if it's unmarked the value is -1.
 - if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5.
 - if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one)
@@ -227,6 +234,21 @@ The server will try to convert the printed message to a float, a failure will gi
         text: write an expression to add x and y.    # optional (default: '')
         correct: path/to/myscript
+An example of a script in python that validades an answer is
+
+    #!/usr/bin/env python3.4
+
+    import sys
+    s = sys.stdin.read()
+    if s == 'Alibaba':
+        print(1.0)
+    else:
+        print(0.0)
+    exit(0)
+
+but any script language or executable program can be used for this purpose.
+
+
 ### Generator
 A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!).
@@ -236,6 +258,22 @@ A generator question will run an external program that is expected to print a qu
         type: generator
         script: path/to/generator_script
+An example of a question generator is the following
+
+    :::python linenums="True"
+    #!/usr/bin/env python3.4
+    from random import randint
+
+    x = randint(10,20)
+    y = randint(10,20)
+    s = '''
+    ref: addition
+    type: text
+    text: How much is {0} plus {1}?
+    correct: {2}
+    '''.format(x, y, x + y)
+    print(s)
+
 ## Writing good looking questions
 The text of the questions (and options in radio and checkbox type questios) is parsed as markdown and code is prettyfied using Pygments. Equations can be inserted like in LaTeX and are rendered using MathJax.
@@ -243,12 +281,12 @@ The text of the questions (and options in radio and checkbox type questios) is p
 A good way to define multiple lines of text in the questions is to use the bar |. Yaml will use all the text that is indented to the right of that column. Example
         text:   |
-                Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX 
+                Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX
                 and pretty code in several languages
-                
+
                 ```.C
                 int main(){
                     return 0;
                 }
                 ```
-
+        # this line does stops the text because it is not indented
@@ -34,7 +34,11 @@ class Database(object):
             c.execute('INSERT INTO tests VALUES (?,?,?,?,?)', values)
             # store grade of every question in the test
-            ans = [(t['ref'], q['ref'], t['number'], q['grade'], str(t['finish_time'])) for q in t['questions']]
+            try:
+                ans = [(t['ref'], q['ref'], t['number'], q['grade'], str(t['finish_time'])) for q in t['questions']]
+            except KeyError as e:
+                print('  * Questions {0} do not have grade defined.'.format(tuple(q['ref'] for q in t['questions'] if 'grade' not in q)))
+                raise e
             c.executemany('INSERT INTO questions VALUES (?,?,?,?,?)', ans)
     def student_reset_pw(self, d):
@@ -34,6 +34,10 @@ class QuestionsPool(dict):
         # add defaults if missing from sources
         for i, q in enumerate(questions):
+            if not isinstance(q, dict):
+                print('  * question with index {0} in file {1} is not a dict. Ignoring...'.format(i, filename))
+                continue
+
             # filename and index (number in the file, 0 based)
             q['filename'] = filename
             q['index'] = i
@@ -74,8 +78,11 @@ def create_question(q):
         'checkbox'  : QuestionCheckbox,
         'text'      : QuestionText,
         'text_regex': QuestionTextRegex,
+        'text-regex': QuestionTextRegex,
+        'regex'     : QuestionTextRegex,
         'textarea'  : QuestionTextArea,
         'information': QuestionInformation,
+        'info'      : QuestionInformation,
         ''          : QuestionInformation,  # default
     }
@@ -83,7 +90,7 @@ def create_question(q):
     try:
         questiontype = types[q['type']]
     except KeyError:
-        print('    * unsupported question type in "%s:%s".' % (q['filename'], q['ref']))
+        print('  * unsupported question type in "%s:%s".' % (q['filename'], q['ref']))
         questiontype = Question
     # create question instance and return
@@ -94,9 +101,14 @@ def create_question(q):
 def question_generator(q):
     '''Run an external script that will generate a question in yaml format.
     This function will return the yaml converted back to a dict.'''
-    # raise exception('question generation not yet implemented.')
+
     q['stdin'] = q.get('stdin', '')
-    p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+
+    try:
+        p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+    except FileNotFoundError:
+        print('  * Script {0} defined in question {1} could not be found'.format(q['script'], q['ref']))
+
     try:
         qyaml = p.communicate(input=q['stdin'].encode('utf-8'), timeout=5)[0].decode('utf-8')
     except subprocess.TimeoutExpired:
@@ -112,7 +124,9 @@ class Question(dict):
     to a student.
     Instances can shuffle options, or automatically generate questions.
     '''
-    pass
+    def correct(self):
+        self['grade'] = 0.0
+        return 0.0
 # ===========================================================================
@@ -355,7 +369,12 @@ class QuestionTextArea(Question):
             # The correction program expects data from stdin and prints the result to stdout.
             # The result should be a string that can be parsed to a float.
-            p = subprocess.Popen([self['correct']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+            try:
+                p = subprocess.Popen([self['correct']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+            except FileNotFoundError as e:
+                print('  * Script "{0}" defined in question "{1}" could not be found'.format(self['correct'], self['ref']))
+                raise e
+
             try:
                 value = p.communicate(input=self['answer'].encode('utf-8'), timeout=5)[0].decode('utf-8')  # esta a dar erro!
             except subprocess.TimeoutExpired:
@@ -369,17 +388,8 @@ class QuestionTextArea(Question):
             try:
                 self['grade'] = float(value)
             except (ValueError):
-                cherrypy.log.error('While checking answer, process %s returned a non float value: %s' % (self['correct'], value), 'APPLICATION')
                 self['grade'] = 0.0
-
-            # Example script to validade answers:
-            #     import sys
-            #     s = sys.stdin.read()
-            #     if s=='Alibaba':
-            #         print(1.0)
-            #     else:
-            #         print(0.0)
-            #     exit(0)
+                raise Exception('Correction of question "%s" returned nonfloat.' % self['ref'])
         return self['grade']
@@ -111,17 +111,20 @@ class Root(object):
         # store the answers in the Test, correct it, save JSON and
         # store results in the database
         t.update_answers(ans)
+        # try:
         t.correct()
+        # except:
+        #     cherrypy.log.error('Failed to correct test of student %s' % t['number'], 'APPLICATION')
+        #     t['grade'] = None
         if t['save_answers']:
             t.save_json(self.testconf['answers_dir'])
         self.database.save_test(t)
-        if t['practice_mode']:
+        if t['practice']:
             raise cherrypy.HTTPRedirect('/test')
         else:
-
             # ---- Expire session ----
             self.loggedin.discard(t['number'])
             cherrypy.lib.sessions.expire()  # session coockie expires client side
@@ -146,7 +149,7 @@ def parse_arguments():
         help='Show hints in questions, if available')
     argparser.add_argument('--save_answers', action='store_true',
         help='Saves answers in JSON format')
-    argparser.add_argument('--practice_mode', action='store_true',
+    argparser.add_argument('--practice', action='store_true',
         help='Show correction results and allow repetitive resubmission of the test')
     argparser.add_argument('testfile', type=str, nargs='+', help='test in YAML format.')
     return argparser.parse_args()
@@ -155,7 +158,7 @@ def parse_arguments():
 if __name__ == '__main__':
     # --- parse command line arguments and build base test
     arg = parse_arguments()
-    testconf = test.read_configuration(arg.testfile[0], debug=arg.debug, show_points=arg.show_points, show_hints=arg.show_hints, save_answers=arg.save_answers, practice_mode=arg.practice_mode)
+    testconf = test.read_configuration(arg.testfile[0], debug=arg.debug, show_points=arg.show_points, show_hints=arg.show_hints, save_answers=arg.save_answers, practice=arg.practice)
     print('=' * 79)
     print('- Title:      %s' % testconf['title'])
@@ -105,6 +105,7 @@
             <%!
                 import markdown as md
                 import yaml
+                import random
             %>
             <%def name="pretty(text)">
                 ${md.markdown(str(text), extensions=['markdown.extensions.tables',
@@ -122,7 +123,7 @@
                 </pre>
             % endif
-            % if t['practice_mode'] and 'grade' in t:
+            % if t['practice'] and 'grade' in t:
                 <div class="jumbotron drop-shadow">
                     <h1>Resultado</h1>
                     <p>Teve <strong>${'{:.1f}'.format(t['grade'])}</strong> valores no teste.</p>
@@ -217,7 +218,7 @@
                                 % endif  # modal
                             % endif  # show_hints
-                            % if t['practice_mode'] and 'grade' in q:
+                            % if t['practice'] and 'grade' in q:
                                 % if q['grade'] > 0.99:
                                     <div class="alert alert-success" role="alert">
                                         <span class="glyphicon glyphicon-ok" aria-hidden="true"></span>
@@ -232,6 +233,9 @@
                                     <div class="alert alert-danger" role="alert">
                                         <span class="glyphicon glyphicon-remove" aria-hidden="true"></span>
                                         ${round(q['grade'] * q['points'] / total_points * 20.0, 1)} pontos
+                                        <p>
+                                            ${random.choice(t['offensive']) if 'offensive' in t else ''}
+                                        </p>
                                     </div>
                                 % endif
                             % endif
@@ -261,7 +265,6 @@
         </form>
     </div>
-
     <!-- Modal de confirmacao -->
     <div class="modal fade" id="confirmar" tabindex="-1" role="dialog" aria-labelledby="myModalLabel" aria-hidden="true">
       <div class="modal-dialog">
@@ -9,7 +9,7 @@ import questions
 import database
 # ============================================================================
-def read_configuration(filename, debug=False, show_points=False, show_hints=False, practice_mode=False, save_answers=False):
+def read_configuration(filename, debug=False, show_points=False, show_hints=False, practice=False, save_answers=False):
     # FIXME validar se ficheiros e directorios existem???
     if not os.path.isfile(filename):
         print('Cannot find file "%s"' % filename)
@@ -23,7 +23,7 @@ def read_configuration(filename, debug=False, show_points=False, show_hints=Fals
     test['title'] = str(test.get('title', ''))
     test['show_hints'] = bool(test.get('show_hints', show_hints))
     test['show_points'] = bool(test.get('show_points', show_points))
-    test['practice_mode'] = bool(test.get('practice_mode', practice_mode))
+    test['practice'] = bool(test.get('practice', practice))
     test['debug'] = bool(test.get('debug', debug))
     test['save_answers'] = bool(test.get('save_answers', save_answers))
     if test['save_answers']:
	@@ -11,6 +11,7 @@		@@ -11,6 +11,7 @@
11		11
12	# TODO	12	# TODO
13		13
		14	+- Quando apresenta o teste, preencher com os valores definidos em answer (permite que professor dê informação à partida, e no modo practice fiquem com o preenchido anteriormente)
14	- testar envio de parametros para stdin para perguntas tipo generator	15	- testar envio de parametros para stdin para perguntas tipo generator
15	- permitir enviar varios testes, aluno escolhe qual o teste que quer fazer.	16	- permitir enviar varios testes, aluno escolhe qual o teste que quer fazer.
16	- criar script json2md.py ou outra forma de gerar um teste ja realizado	17	- criar script json2md.py ou outra forma de gerar um teste ja realizado
	@@ -9,7 +9,7 @@ import questions		@@ -9,7 +9,7 @@ import questions
9	import database	9	import database
10		10
11	# ============================================================================	11	# ============================================================================
12	-def read_configuration(filename, debug=False, show_points=False, show_hints=False, practice_mode=False, save_answers=False):	12	+def read_configuration(filename, debug=False, show_points=False, show_hints=False, practice=False, save_answers=False):
13	# FIXME validar se ficheiros e directorios existem???	13	# FIXME validar se ficheiros e directorios existem???
14	if not os.path.isfile(filename):	14	if not os.path.isfile(filename):
15	print('Cannot find file "%s"' % filename)	15	print('Cannot find file "%s"' % filename)
	@@ -23,7 +23,7 @@ def read_configuration(filename, debug=False, show_points=False, show_hints=Fals		@@ -23,7 +23,7 @@ def read_configuration(filename, debug=False, show_points=False, show_hints=Fals
23	test['title'] = str(test.get('title', ''))	23	test['title'] = str(test.get('title', ''))
24	test['show_hints'] = bool(test.get('show_hints', show_hints))	24	test['show_hints'] = bool(test.get('show_hints', show_hints))
25	test['show_points'] = bool(test.get('show_points', show_points))	25	test['show_points'] = bool(test.get('show_points', show_points))
26	- test['practice_mode'] = bool(test.get('practice_mode', practice_mode))	26	+ test['practice'] = bool(test.get('practice', practice))
27	test['debug'] = bool(test.get('debug', debug))	27	test['debug'] = bool(test.get('debug', debug))
28	test['save_answers'] = bool(test.get('save_answers', save_answers))	28	test['save_answers'] = bool(test.get('save_answers', save_answers))
29	if test['save_answers']:	29	if test['save_answers']: