Miguel Barão / perguntations

Download as
Browse Code »

Commit 1b9491c2682cde0625d2dd4305c340375f6d5ea8

Authored by Miguel Barão
1 parent ffb53a93
Exists in master and in 1 other branch dev

- use schema to validade test. 

- removed option --show_ref, now depends on debug.
- the option --correct is not implemented yet.
Showing 14 changed files with 218 additions and 226 deletions   Show diff stats
demo/demo.yaml
  Diff comments   View file @1b9491c
... ... @@ -14,9 +14,6 @@ database: students.db
14 14 # Directory where the submitted and corrected test are stored for later review.
15 15 answers_dir: ans
16 16  
17   -# Server used to compile & execute code
18   -jobe_server: 192.168.1.85
19   -
20 17 # --- optional settings: -----------------------------------------------------
21 18  
22 19 # Title of this test, e.g. course name, year or test number
... ... @@ -35,49 +32,52 @@ autosubmit: false
35 32 # shown to the student. If false, the test is saved but not corrected.
36 33 # No grade is shown to the student.
37 34 # (default: true)
38   -autocorrect: true
  35 +autocorrect: false
39 36  
40 37 # Show points for each question (min and max).
41 38 # (default: true)
42 39 show_points: true
43 40  
44   -# scale final grade to an interval, e.g. [0, 20], keeping the relative weight
45   -# of the points declared in the questions below.
  41 +# Scale the points of the questions so that the final grade is in the given
  42 +# interval.
46 43 # (default: no scaling, just use question points)
47   -scale: [0, 5]
  44 +scale: [0, 20]
48 45  
49 46  
50 47 # ----------------------------------------------------------------------------
51   -# Base path applied to the questions files and all the scripts
52   -# including question generators and correctors.
53   -# Either absolute path or relative to current directory can be used.
54   -questions_dir: .
55   -
56   -# (optional) List of files containing questions in yaml format.
57   -# Selected questions will be obtained from these files.
58   -# If undefined, all yaml files in questions_dir are loaded (not recommended).
  48 +# Files to import. Each file contains a list of questions in yaml format.
59 49 files:
60 50 - questions/questions-tutorial.yaml
61 51  
62 52 # This is the list of questions that will make up the test.
63 53 # The order is preserved.
64   -# There are several ways to define each question (explained below).
  54 +# Each question is a dictionary with a question `ref` or a list of `ref`.
  55 +# If a list is given, one question will be choosen randomly to each student.
  56 +# The `points` for each question is optional and is 1.0 by default for normal
  57 +# questions. Informative type of "questions" will have 0.0 points.
  58 +# Points are automatically scaled if `scale` key is defined.
65 59 questions:
66 60 - ref: tut-test
67   - - tut-questions
  61 + - ref: tut-questions
  62 +
  63 + # these will have 1.0 points
  64 + - ref: tut-radio
  65 + - ref: tut-checkbox
  66 + - ref: tut-text
  67 + - ref: tut-text-regex
  68 + - ref: tut-numeric-interval
68 69  
69   - - tut-radio
70   - - tut-checkbox
71   - - tut-text
72   - - tut-text-regex
73   - - tut-numeric-interval
  70 + # this question will have 2.0 points
74 71 - ref: tut-textarea
75 72 points: 2.0
76 73  
77   - - tut-information
78   - - tut-success
79   - - tut-warning
80   - - [tut-alert1, tut-alert2]
81   - - tut-generator
82   - - tut-yamllint
83   - # - tut-code
  74 + # these will have 0.0 points:
  75 + - ref: tut-information
  76 + - ref: tut-success
  77 + - ref: tut-warning
  78 +
  79 + # choose one from the list:
  80 + - ref: [tut-alert1, tut-alert2]
  81 +
  82 + - ref: tut-generator
  83 + - ref: tut-yamllint
... ...
demo/questions/questions-tutorial.yaml
  Diff comments   View file @1b9491c
... ... @@ -20,24 +20,20 @@
20 20 database: students.db # base de dados previamente criada com initdb
21 21 answers_dir: ans # directório onde ficam os testes dos alunos
22 22  
23   - # opcional
  23 + # opcionais
24 24 duration: 60 # duração da prova em minutos (default: inf)
25 25 autosubmit: true # submissão automática (default: false)
26 26 show_points: true # mostra cotação das perguntas (default: true)
27   - scale: [0, 20] # limites inferior e superior da escala (default: [0,20])
28   - scale_points: true # normaliza cotações para a escala definida
29   - jobe_server: moodle-jobe.uevora.pt # server used to compile & execute code
30   - debug: false # mostra informação de debug no browser
  27 + scale: [0, 20] # normaliza cotações para o intervalo indicado.
  28 + # não normaliza por defeito (default: None)
31 29  
32 30 # --------------------------------------------------------------------------
33   - questions_dir: ~/topics # raíz da árvore de directórios das perguntas
34   -
35 31 # Ficheiros de perguntas a importar (relativamente a `questions_dir`)
36 32 files:
37 33 - tabelas.yaml
38   - - topic_A/questions.yaml
39   - - topic_B/part_1/questions.yaml
40   - - topic_B/part_2/questions.yaml
  34 + - topic1/questions.yaml
  35 + - topic2/part1/questions.yaml
  36 + - topic2/part2/questions.yaml
41 37  
42 38 # --------------------------------------------------------------------------
43 39 # Especificação das perguntas do teste e respectivas cotações.
... ... @@ -50,13 +46,10 @@
50 46 - ref: pergunta2
51 47 points: 2.0
52 48  
53   - # a cotação é 1.0 por defeito
  49 + # por defeinto, a cotação da pergunta é 1.0 valor
54 50 - ref: pergunta3
55 51  
56   - # uma string (não dict), é interpretada como referência
57   - - tabela-auxiliar
58   -
59   - # escolhe aleatoriamente uma das variantes
  52 + # escolhe aleatoriamente uma das variantes da pergunta
60 53 - ref: [pergunta3a, pergunta3b]
61 54 points: 0.5
62 55  
... ... @@ -96,8 +89,7 @@
96 89 text: |
97 90 Quando o texto da pergunta tem várias linhas, dá jeito usar o símbolo
98 91 `|` de pipe, para indicar que tudo o que estiver indentado faz parte do
99   - texto.
100   - É o caso desta pergunta.
  92 + texto. É o caso desta pergunta.
101 93  
102 94 O texto das perguntas é escrito em `markdown` e suporta fórmulas em
103 95 LaTeX.
... ... @@ -105,8 +97,8 @@
105 97 #---------------------------------------------------------------------------
106 98 ```
107 99  
108   - As chaves são usadas para construir o teste e não se podem repetir, mesmo em
109   - ficheiros diferentes.
  100 + As chaves são usadas para construir o teste e não se podem repetir, mesmo
  101 + em ficheiros diferentes.
110 102 De seguida mostram-se exemplos dos vários tipos de perguntas.
111 103  
112 104 # ----------------------------------------------------------------------------
... ...
mypy.ini
  Diff comments   View file @1b9491c
1 1 [mypy]
2 2 python_version = 3.9
  3 +ignore_missing_imports = True
  4 +
3 5  
4 6 ; [mypy-setuptools.*]
5 7 ; ignore_missing_imports = True
... ...
perguntations/__init__.py
  Diff comments   View file @1b9491c
1   -# Copyright (C) 2021 Miguel Barão
  1 +# Copyright (C) 2022 Miguel Barão
2 2 #
3 3 # THE MIT License
4 4 #
... ... @@ -32,10 +32,10 @@ proof of submission and for review.
32 32 '''
33 33  
34 34 APP_NAME = 'perguntations'
35   -APP_VERSION = '2021.09.dev1'
  35 +APP_VERSION = '2022.01.dev1'
36 36 APP_DESCRIPTION = __doc__
37 37  
38 38 __author__ = 'Miguel Barão'
39   -__copyright__ = 'Copyright 2021, Miguel Barão'
  39 +__copyright__ = 'Copyright 2022, Miguel Barão'
40 40 __license__ = 'MIT license'
41 41 __version__ = APP_VERSION
... ...
perguntations/app.py
  Diff comments   View file @1b9491c
... ... @@ -55,6 +55,7 @@ class App():
55 55  
56 56 # ------------------------------------------------------------------------
57 57 def __init__(self, config):
  58 + self.debug = config['debug']
58 59 self._make_test_factory(config['testfile'])
59 60 self._db_setup() # setup engine and load all students
60 61  
... ... @@ -126,10 +127,9 @@ class App():
126 127 logger.warning('"%s" does not exist', uid)
127 128 return 'nonexistent'
128 129  
129   -
130 130 if uid != '0' and self._students[uid]['state'] != 'allowed':
131 131 logger.warning('"%s" login not allowed', uid)
132   - return 'not allowed'
  132 + return 'not_allowed'
133 133  
134 134 if hashed == '': # set password on first login
135 135 await self.set_password(uid, password)
... ...
perguntations/main.py
  Diff comments   View file @1b9491c
... ... @@ -20,7 +20,7 @@ from perguntations.tools import load_yaml
20 20 from perguntations import APP_NAME, APP_VERSION
21 21  
22 22 # ----------------------------------------------------------------------------
23   -def parse_cmdline_arguments():
  23 +def parse_cmdline_arguments() -> argparse.Namespace:
24 24 '''
25 25 Get command line arguments
26 26 '''
... ... @@ -40,9 +40,6 @@ def parse_cmdline_arguments():
40 40 parser.add_argument('--debug',
41 41 action='store_true',
42 42 help='Enable debug messages')
43   - parser.add_argument('--show-ref',
44   - action='store_true',
45   - help='Show question references')
46 43 parser.add_argument('--review',
47 44 action='store_true',
48 45 help='Review mode: doesn\'t generate test')
... ... @@ -59,7 +56,6 @@ def parse_cmdline_arguments():
59 56 help='Show version information and exit')
60 57 return parser.parse_args()
61 58  
62   -
63 59 # ----------------------------------------------------------------------------
64 60 def get_logger_config(debug=False) -> dict:
65 61 '''
... ... @@ -120,10 +116,9 @@ def main():
120 116 # --- start application --------------------------------------------------
121 117 config = {
122 118 'testfile': args.testfile,
123   - 'debug': args.debug,
124 119 'allow_all': args.allow_all,
125 120 'allow_list': args.allow_list,
126   - 'show_ref': args.show_ref,
  121 + 'debug': args.debug,
127 122 'review': args.review,
128 123 'correct': args.correct,
129 124 }
... ...
perguntations/serve.py
  Diff comments   View file @1b9491c
... ... @@ -93,6 +93,11 @@ class BaseHandler(tornado.web.RequestHandler):
93 93 '''simplifies access to the application a little bit'''
94 94 return self.application.testapp
95 95  
  96 + # @property
  97 + # def debug(self) -> bool:
  98 + # '''check if is running in debug mode'''
  99 + # return self.application.testapp.debug
  100 +
96 101 def get_current_user(self):
97 102 '''
98 103 Since HTTP is stateless, a cookie is used to identify the user.
... ... @@ -112,7 +117,7 @@ class LoginHandler(BaseHandler):
112 117 _prefix = re.compile(r'[a-z]')
113 118 _error_msg = {
114 119 'wrong_password': 'Senha errada',
115   - 'not allowed': 'Não está autorizado a fazer o teste',
  120 + 'not_allowed': 'Não está autorizado a fazer o teste',
116 121 'nonexistent': 'Número de aluno inválido'
117 122 }
118 123  
... ... @@ -195,7 +200,7 @@ class RootHandler(BaseHandler):
195 200 test = self.testapp.get_test(uid)
196 201 name = self.testapp.get_name(uid)
197 202 self.render('test.html', t=test, uid=uid, name=name, md=md_to_html,
198   - templ=self._templates)
  203 + templ=self._templates, debug=self.testapp.debug)
199 204  
200 205 # --- POST
201 206 @tornado.web.authenticated
... ... @@ -448,8 +453,8 @@ class ReviewHandler(BaseHandler):
448 453  
449 454 uid = test['student']
450 455 name = self.testapp.get_name(uid)
451   - self.render('review.html', t=test, uid=uid, name=name,
452   - md=md_to_html, templ=self._templates)
  456 + self.render('review.html', t=test, uid=uid, name=name, md=md_to_html,
  457 + templ=self._templates, debug=self.testapp.debug)
453 458  
454 459  
455 460 # ----------------------------------------------------------------------------
... ...
perguntations/templates/question-information.html
  Diff comments   View file @1b9491c
... ... @@ -17,9 +17,9 @@
17 17 {{ md(q['text']) }}
18 18 </div>
19 19  
20   - {% if show_ref %}
  20 + {% if debug %}
21 21 <hr>
22 22 file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
23 23 ref: <code>{{ q['ref'] }}</code>
24 24 {% end %}
25   -</div>
26 25 \ No newline at end of file
  26 +</div>
... ...
perguntations/templates/question.html
  Diff comments   View file @1b9491c
... ... @@ -29,11 +29,11 @@
29 29 </p>
30 30 </div>
31 31  
32   - {% if show_ref %}
  32 + {% if debug %}
33 33 <div class="card-footer">
34 34 file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
35 35 ref: <code>{{ q['ref'] }}</code>
36 36 </div>
37 37 {% end %}
38 38 </div>
39   -{% end %}
40 39 \ No newline at end of file
  40 +{% end %}
... ...
perguntations/templates/review-question-information.html
  Diff comments   View file @1b9491c
... ... @@ -16,9 +16,9 @@
16 16 <div id="text">
17 17 {{ md(q['text']) }}
18 18 </div>
19   - {% if t['show_ref'] %}
  19 + {% if debug %}
20 20 <hr>
21 21 file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
22 22 ref: <code>{{ q['ref'] }}</code>
23 23 {% end %}
24   -</div>
25 24 \ No newline at end of file
  25 +</div>
... ...
perguntations/templates/review-question.html
  Diff comments   View file @1b9491c
... ... @@ -65,7 +65,7 @@
65 65 {% end %}
66 66 {% end %}
67 67  
68   - {% if t['show_ref'] %}
  68 + {% if debug %}
69 69 <hr>
70 70 file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
71 71 ref: <code>{{ q['ref'] }}</code>
... ... @@ -109,7 +109,7 @@
109 109 {% end %}
110 110 </p>
111 111  
112   - {% if t['show_ref'] %}
  112 + {% if debug %}
113 113 <hr>
114 114 file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
115 115 ref: <code>{{ q['ref'] }}</code>
... ... @@ -118,4 +118,4 @@
118 118 </div> <!-- card-footer -->
119 119 </div> <!-- card -->
120 120 {% end %} <!-- if answer not None -->
121   -{% end %} <!-- block -->
122 121 \ No newline at end of file
  122 +{% end %} <!-- block -->
... ...
perguntations/templates/review.html
  Diff comments   View file @1b9491c
... ... @@ -113,7 +113,7 @@
113 113 </div> <!-- jumbotron -->
114 114  
115 115 {% for i, q in enumerate(t['questions']) %}
116   - {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), t=t) %}
  116 + {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), t=t, debug=debug) %}
117 117 {% end %}
118 118  
119 119 </div> <!-- container -->
... ...
perguntations/templates/test.html
  Diff comments   View file @1b9491c
... ... @@ -114,7 +114,7 @@
114 114 {% module xsrf_form_html() %}
115 115  
116 116 {% for i, q in enumerate(t['questions']) %}
117   - {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), show_ref=t['show_ref']) %}
  117 + {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), debug=debug) %}
118 118 {% end %}
119 119  
120 120 <div class="form-row">
... ...
perguntations/testfactory.py
  Diff comments   View file @1b9491c
... ... @@ -6,8 +6,9 @@ TestFactory - generates tests for students
6 6 from os import path
7 7 import random
8 8 import logging
9   -import re
10   -from typing import TypedDict
  9 +
  10 +# other libraries
  11 +import schema
11 12  
12 13 # this project
13 14 from perguntations.questions import QFactory, QuestionException, QDict
... ... @@ -17,10 +18,49 @@ from perguntations.tools import load_yaml
17 18 # Logger configuration
18 19 logger = logging.getLogger(__name__)
19 20  
20   -ConfigDict = TypedDict('ConfigDict', {
21   - 'title': str
22   - # TODO add other fields
23   - })
  21 +# --- test validation --------------------------------------------------------
  22 +def check_answers_directory(ans: str) -> bool:
  23 + '''Checks is answers_dir exists and is writable'''
  24 + testfile = path.join(path.expanduser(ans), 'REMOVE-ME')
  25 + try:
  26 + with open(testfile, 'w', encoding='utf-8') as file:
  27 + file.write('You can safely remove this file.')
  28 + except OSError:
  29 + return False
  30 + return True
  31 +
  32 +def check_import_files(files: list) -> bool:
  33 + '''Checks if the question files exist'''
  34 + if not files:
  35 + return False
  36 + for file in files:
  37 + if not path.isfile(file):
  38 + return False
  39 + return True
  40 +
  41 +def normalize_question_list(questions: list) -> None:
  42 + '''convert question ref from string to list of string'''
  43 + for question in questions:
  44 + if isinstance(question['ref'], str):
  45 + question['ref'] = [question['ref']]
  46 +
  47 +test_schema = schema.Schema({
  48 + 'ref': schema.Regex('^[a-zA-Z0-9_-]+$'),
  49 + 'database': schema.And(str, path.isfile),
  50 + 'answers_dir': schema.And(str, check_answers_directory),
  51 + 'title': str,
  52 + schema.Optional('duration'): int,
  53 + schema.Optional('autosubmit'): bool,
  54 + schema.Optional('autocorrect'): bool,
  55 + schema.Optional('show_points'): bool,
  56 + schema.Optional('scale'): schema.And([schema.Use(float)],
  57 + lambda s: len(s) == 2),
  58 + 'files': schema.And([str], check_import_files),
  59 + 'questions': [{
  60 + 'ref': schema.Or(str, [str]),
  61 + schema.Optional('points'): float
  62 + }]
  63 + }, ignore_extra_keys=True)
24 64  
25 65 # ============================================================================
26 66 class TestFactoryException(Exception):
... ... @@ -36,35 +76,31 @@ class TestFactory(dict):
36 76 '''
37 77  
38 78 # ------------------------------------------------------------------------
39   - def __init__(self, conf: ConfigDict) -> None:
  79 + def __init__(self, conf) -> None:
40 80 '''
41 81 Loads configuration from yaml file, then overrides some configurations
42 82 using the conf argument.
43 83 Base questions are added to a pool of questions factories.
44 84 '''
45 85  
  86 + test_schema.validate(conf)
  87 +
46 88 # --- set test defaults and then use given configuration
47 89 super().__init__({ # defaults
48   - 'title': '',
49 90 'show_points': True,
50 91 'scale': None,
51 92 'duration': 0, # 0=infinite
52 93 'autosubmit': False,
53 94 'autocorrect': True,
54   - # 'debug': False, # FIXME not property of a test...
55   - 'show_ref': False,
56 95 })
57 96 self.update(conf)
  97 + normalize_question_list(self['questions'])
58 98  
59 99 # --- for review, we are done. no factories needed
60 100 # if self['review']: FIXME
61 101 # logger.info('Review mode. No questions loaded. No factories.')
62 102 # return
63 103  
64   - # --- perform sanity checks and normalize the test questions
65   - self.sanity_checks()
66   - logger.info('Sanity checks PASSED.')
67   -
68 104 # --- find refs of all questions used in the test
69 105 qrefs = {r for qq in self['questions'] for r in qq['ref']}
70 106 logger.info('Declared %d questions (each test uses %d).',
... ... @@ -74,7 +110,7 @@ class TestFactory(dict):
74 110 self['question_factory'] = {}
75 111  
76 112 for file in self["files"]:
77   - fullpath = path.normpath(path.join(self["questions_dir"], file))
  113 + fullpath = path.normpath(file)
78 114  
79 115 logger.info('Loading "%s"...', fullpath)
80 116 questions = load_yaml(fullpath) # , default=[])
... ... @@ -85,7 +121,7 @@ class TestFactory(dict):
85 121 msg = f'Question {i} in {file} is not a dictionary'
86 122 raise TestFactoryException(msg)
87 123  
88   - # check if ref is missing, then set to '/path/file.yaml:3'
  124 + # check if ref is missing, then set to '//file.yaml:3'
89 125 if 'ref' not in question:
90 126 question['ref'] = f'{file}:{i:04}'
91 127 logger.warning('Missing ref set to "%s"', question["ref"])
... ... @@ -115,103 +151,104 @@ class TestFactory(dict):
115 151  
116 152  
117 153 # ------------------------------------------------------------------------
118   - def check_test_ref(self) -> None:
119   - '''Test must have a `ref`'''
120   - if 'ref' not in self:
121   - raise TestFactoryException('Missing "ref" in configuration!')
122   - if not re.match(r'^[a-zA-Z0-9_-]+$', self['ref']):
123   - raise TestFactoryException('Test "ref" can only contain the '
124   - 'characters a-zA-Z0-9_-')
125   -
126   - def check_missing_database(self) -> None:
127   - '''Test must have a database'''
128   - if 'database' not in self:
129   - raise TestFactoryException('Missing "database" in configuration')
130   - if not path.isfile(path.expanduser(self['database'])):
131   - msg = f'Database "{self["database"]}" not found!'
132   - raise TestFactoryException(msg)
133   -
134   - def check_missing_answers_directory(self) -> None:
135   - '''Test must have a answers directory'''
136   - if 'answers_dir' not in self:
137   - msg = 'Missing "answers_dir" in configuration'
138   - raise TestFactoryException(msg)
139   -
140   - def check_answers_directory_writable(self) -> None:
141   - '''Answers directory must be writable'''
142   - testfile = path.join(path.expanduser(self['answers_dir']), 'REMOVE-ME')
143   - try:
144   - with open(testfile, 'w', encoding='utf-8') as file:
145   - file.write('You can safely remove this file.')
146   - except OSError as exc:
147   - msg = f'Cannot write answers to directory "{self["answers_dir"]}"'
148   - raise TestFactoryException(msg) from exc
149   -
150   - def check_questions_directory(self) -> None:
151   - '''Check if questions directory is missing or not accessible.'''
152   - if 'questions_dir' not in self:
153   - logger.warning('Missing "questions_dir". Using "%s"',
154   - path.abspath(path.curdir))
155   - self['questions_dir'] = path.curdir
156   - elif not path.isdir(path.expanduser(self['questions_dir'])):
157   - raise TestFactoryException(f'Can\'t find questions directory '
158   - f'"{self["questions_dir"]}"')
159   -
160   - def check_import_files(self) -> None:
161   - '''Check if there are files to import (with questions)'''
162   - if 'files' not in self:
163   - msg = ('Missing "files" in configuration with the list of '
164   - 'question files to import!')
165   - raise TestFactoryException(msg)
166   -
167   - if isinstance(self['files'], str):
168   - self['files'] = [self['files']]
169   -
170   - def check_question_list(self) -> None:
171   - '''normalize question list'''
172   - if 'questions' not in self:
173   - raise TestFactoryException('Missing "questions" in configuration')
174   -
175   - for i, question in enumerate(self['questions']):
176   - # normalize question to a dict and ref to a list of references
177   - if isinstance(question, str): # e.g., - some_ref
178   - question = {'ref': [question]} # becomes - ref: [some_ref]
179   - elif isinstance(question, dict) and isinstance(question['ref'], str):
180   - question['ref'] = [question['ref']]
181   - elif isinstance(question, list):
182   - question = {'ref': [str(a) for a in question]}
183   -
184   - self['questions'][i] = question
185   -
186   - def check_missing_title(self) -> None:
187   - '''Warns if title is missing'''
188   - if not self['title']:
189   - logger.warning('Title is undefined!')
190   -
191   - def check_grade_scaling(self) -> None:
192   - '''Just informs the scale limits'''
193   - if 'scale_points' in self:
194   - msg = ('*** DEPRECATION WARNING: *** scale_points, scale_min, '
195   - 'scale_max were replaced by "scale: [min, max]".')
196   - logger.warning(msg)
197   - self['scale'] = [self['scale_min'], self['scale_max']]
  154 + # def check_test_ref(self) -> None:
  155 + # '''Test must have a `ref`'''
  156 + # if 'ref' not in self:
  157 + # raise TestFactoryException('Missing "ref" in configuration!')
  158 + # if not re.match(r'^[a-zA-Z0-9_-]+$', self['ref']):
  159 + # raise TestFactoryException('Test "ref" can only contain the '
  160 + # 'characters a-zA-Z0-9_-')
  161 +
  162 + # def check_missing_database(self) -> None:
  163 + # '''Test must have a database'''
  164 + # if 'database' not in self:
  165 + # raise TestFactoryException('Missing "database" in configuration')
  166 + # if not path.isfile(path.expanduser(self['database'])):
  167 + # msg = f'Database "{self["database"]}" not found!'
  168 + # raise TestFactoryException(msg)
  169 +
  170 + # def check_missing_answers_directory(self) -> None:
  171 + # '''Test must have a answers directory'''
  172 + # if 'answers_dir' not in self:
  173 + # msg = 'Missing "answers_dir" in configuration'
  174 + # raise TestFactoryException(msg)
  175 +
  176 + # def check_answers_directory_writable(self) -> None:
  177 + # '''Answers directory must be writable'''
  178 + # testfile = path.join(path.expanduser(self['answers_dir']), 'REMOVE-ME')
  179 + # try:
  180 + # with open(testfile, 'w', encoding='utf-8') as file:
  181 + # file.write('You can safely remove this file.')
  182 + # except OSError as exc:
  183 + # msg = f'Cannot write answers to directory "{self["answers_dir"]}"'
  184 + # raise TestFactoryException(msg) from exc
  185 +
  186 + # def check_questions_directory(self) -> None:
  187 + # '''Check if questions directory is missing or not accessible.'''
  188 + # if 'questions_dir' not in self:
  189 + # logger.warning('Missing "questions_dir". Using "%s"',
  190 + # path.abspath(path.curdir))
  191 + # self['questions_dir'] = path.curdir
  192 + # elif not path.isdir(path.expanduser(self['questions_dir'])):
  193 + # raise TestFactoryException(f'Can\'t find questions directory '
  194 + # f'"{self["questions_dir"]}"')
  195 +
  196 + # def check_import_files(self) -> None:
  197 + # '''Check if there are files to import (with questions)'''
  198 + # if 'files' not in self:
  199 + # msg = ('Missing "files" in configuration with the list of '
  200 + # 'question files to import!')
  201 + # raise TestFactoryException(msg)
  202 +
  203 + # if isinstance(self['files'], str):
  204 + # self['files'] = [self['files']]
  205 +
  206 + # def check_question_list(self) -> None:
  207 + # '''normalize question list'''
  208 + # if 'questions' not in self:
  209 + # raise TestFactoryException('Missing "questions" in configuration')
  210 +
  211 + # for i, question in enumerate(self['questions']):
  212 + # # normalize question to a dict and ref to a list of references
  213 + # if isinstance(question, str): # e.g., - some_ref
  214 + # logger.warning(f'Question "{question}" should be a dictionary')
  215 + # question = {'ref': [question]} # becomes - ref: [some_ref]
  216 + # elif isinstance(question, dict) and isinstance(question['ref'], str):
  217 + # question['ref'] = [question['ref']]
  218 + # elif isinstance(question, list):
  219 + # question = {'ref': [str(a) for a in question]}
  220 +
  221 + # self['questions'][i] = question
  222 +
  223 + # def check_missing_title(self) -> None:
  224 + # '''Warns if title is missing'''
  225 + # if not self['title']:
  226 + # logger.warning('Title is undefined!')
  227 +
  228 + # def check_grade_scaling(self) -> None:
  229 + # '''Just informs the scale limits'''
  230 + # if 'scale_points' in self:
  231 + # msg = ('*** DEPRECATION WARNING: *** scale_points, scale_min, '
  232 + # 'scale_max were replaced by "scale: [min, max]".')
  233 + # logger.warning(msg)
  234 + # self['scale'] = [self['scale_min'], self['scale_max']]
198 235  
199 236  
200 237 # ------------------------------------------------------------------------
201   - def sanity_checks(self) -> None:
202   - '''
203   - Checks for valid keys and sets default values.
204   - Also checks if some files and directories exist
205   - '''
206   - self.check_test_ref()
207   - self.check_missing_database()
208   - self.check_missing_answers_directory()
209   - self.check_answers_directory_writable()
210   - self.check_questions_directory()
211   - self.check_import_files()
212   - self.check_question_list()
213   - self.check_missing_title()
214   - self.check_grade_scaling()
  238 + # def sanity_checks(self) -> None:
  239 + # '''
  240 + # Checks for valid keys and sets default values.
  241 + # Also checks if some files and directories exist
  242 + # '''
  243 + # self.check_test_ref()
  244 + # self.check_missing_database()
  245 + # self.check_missing_answers_directory()
  246 + # self.check_answers_directory_writable()
  247 + # self.check_questions_directory()
  248 + # self.check_import_files()
  249 + # self.check_question_list()
  250 + # self.check_missing_title()
  251 + # self.check_grade_scaling()
215 252  
216 253 # ------------------------------------------------------------------------
217 254 def check_questions(self) -> None:
... ... @@ -230,42 +267,6 @@ class TestFactory(dict):
230 267  
231 268 if question['type'] == 'textarea':
232 269 _runtests_textarea(qref, question)
233   - # if 'tests_right' in question:
234   - # for tnum, right_answer in enumerate(question['tests_right']):
235   - # try:
236   - # question.set_answer(right_answer)
237   - # question.correct()
238   - # except Exception as exc:
239   - # msg = f'Failed to correct "{qref}"'
240   - # raise TestFactoryException(msg) from exc
241   -
242   - # if question['grade'] == 1.0:
243   - # logger.info(' test %i Ok', tnum)
244   - # else:
245   - # logger.error(' TEST %i IS WRONG!!!', tnum)
246   - # elif 'tests_wrong' in question:
247   - # for tnum, wrong_answer in enumerate(question['tests_wrong']):
248   - # try:
249   - # question.set_answer(wrong_answer)
250   - # question.correct()
251   - # except Exception as exc:
252   - # msg = f'Failed to correct "{qref}"'
253   - # raise TestFactoryException(msg) from exc
254   -
255   - # if question['grade'] < 1.0:
256   - # logger.info(' test %i Ok', tnum)
257   - # else:
258   - # logger.error(' TEST %i IS WRONG!!!', tnum)
259   - # else:
260   - # try:
261   - # question.set_answer('')
262   - # question.correct()
263   - # except Exception as exc:
264   - # msg = f'Failed to correct "{qref}"'
265   - # raise TestFactoryException(msg) from exc
266   - # else:
267   - # logger.info(' correct Ok but no tests to run')
268   -
269 270 # ------------------------------------------------------------------------
270 271 async def generate(self):
271 272 '''
... ... @@ -323,11 +324,8 @@ class TestFactory(dict):
323 324 logger.error('%s errors found!', nerr)
324 325  
325 326 # copy these from the test configuratoin to each test instance
326   - inherit = {'ref', 'title', 'database', 'answers_dir',
327   - 'questions_dir', 'files',
328   - 'duration', 'autosubmit', 'autocorrect',
329   - 'scale', 'show_points', 'show_ref'}
330   - # NOT INCLUDED: testfile, allow_all, review, debug
  327 + inherit = ['ref', 'title', 'database', 'answers_dir', 'files', 'scale',
  328 + 'duration', 'autosubmit', 'autocorrect', 'show_points']
331 329  
332 330 return Test({'questions': questions, **{k:self[k] for k in inherit}})
333 331  
... ...