Miguel Barão / aprendizations

Download as
Browse Code »

Commit 8f643ad6a247b911b08270ee1a9cfd9cd02c2a52

Authored by Miguel Barão
1 parent e0d5ddfe
Exists in master and in 1 other branch dev

fix error in radio questions where all options right would break 

added more checks in radio questions
Showing 7 changed files with 141 additions and 87 deletions   Show diff stats
BUGS.md
  Diff comments   View file @8f643ad
1 1  
2 2 # BUGS
3 3  
  4 +- apos clicar no botao responder, inactivar o input (importante quando o tempo de correcção é grande)
4 5 - nao esta a seguir o max_tries definido no ficheiro de dependencias.
5   -- registar last_seen e remover os antigos de cada vez que houver um login.
6 6 - initdb da integrity error se no mesmo comando existirem alunos repetidos (p.ex em ficheiros csv diferentes ou entre csv e opcao -a)
7 7 - double click submits twice.
8   -- marking all options right in a radio question breaks!
9 8 - duplo clicks no botao "responder" dessincroniza as questões, ver debounce em https://stackoverflow.com/questions/20281546/how-to-prevent-calling-of-en-event-handler-twice-on-fast-clicks
10   -- quando termina topico devia apagar as perguntas todas (se falhar a gerar novo topico, aparecem perguntas do antigo)
11   -- apos clicar no botao responder, inactivar o input (importante quando o tempo de correcção é grande)
12 9 - devia mostrar timeout para o aluno saber a razao.
13 10 - permitir configuracao para escolher entre static files locais ou remotos
14 11 - templates question-*.html tem input hidden question_ref que não é usado. remover?
15   -- click numa opcao checkbox fora da checkbox+label não está a funcionar.
16 12 - shift-enter não está a funcionar
17 13 - default prefix should be obtained from each course (yaml conf)?
18   -- tabelas nas perguntas radio/checkbox não ocupam todo o espaço como em question.
19 14  
20 15 # TODO
21 16  
  17 +- checkbox devia ter correct no intervalo [0,1] tal como radio. em caso de desconto a correccção faz 2*x-1. isto permite a mesma semantica nos dois tipos de perguntas.
  18 +- registar last_seen e remover os antigos de cada vez que houver um login.
22 19 - indicar qtos topicos faltam (>=50%) para terminar o curso.
23   -- use run_script_async to run run_script using asyncio.run?
24   -- ao fim de 3 tentativas de login, envial email para aluno com link para definir nova password (com timeout de 5 minutos).
  20 +- ao fim de 3 tentativas com password errada, envia email com nova password.
25 21 - mostrar capitulos e subtopicos de forma hierarquica. clicar no capitulo expande as dependencias.
26 22 - mostrar rankings alunos/perguntas respondidas/% correctas/valor esperado topicos.
27 23 - botão não sei...
... ... @@ -33,14 +29,16 @@
33 29 - tabela com perguntas / quantidade de respostas certas/erradas.
34 30 - tabela com topicos / quantidade de estrelas.
35 31 - pymips: activar/desactivar instruções
36   -- implementar servidor http com redirect para https.
37   -- ao fim de 3 tentativas com password errada, envia email com nova password.
38 32 - titulos das perguntas não suportam markdown.
39 33 - pagina report que permita ver tabela alunos/topicos, estatisticas perguntas mais falhadas, tempo médio por pergunta.
40 34 - normalizar com perguntations.
41 35  
42 36 # FIXED
43 37  
  38 +- marking all options right in a radio question breaks!
  39 +- implementar servidor http com redirect para https.
  40 +- tabelas nas perguntas radio/checkbox não ocupam todo o espaço como em question.
  41 +- click numa opcao checkbox fora da checkbox+label não está a funcionar.
44 42 - mathjax, formulas $$f(x)$$ nas opções de escolha multipla, não ficam centradas em toda a coluna mas apenas na largura do parágrafo.
45 43 - QFactory.generate() devia fazer run da gen_async, ou remover.
46 44 - classificacoes so devia mostrar os que ja fizeram alguma coisa
... ...
aprendizations/learnapp.py
  Diff comments   View file @8f643ad
... ... @@ -16,7 +16,7 @@ import sqlalchemy as sa
16 16  
17 17 # this project
18 18 from .models import Student, Answer, Topic, StudentTopic
19   -from .questions import Question, QFactory, QDict
  19 +from .questions import Question, QFactory, QDict, QuestionException
20 20 from .student import StudentState
21 21 from .tools import load_yaml
22 22  
... ... @@ -100,7 +100,6 @@ class LearnApp(object):
100 100 # if graph has topics that are not in the database, add them
101 101 self.add_missing_topics(self.deps.nodes())
102 102  
103   -
104 103 if check:
105 104 self.sanity_check_questions()
106 105  
... ... @@ -113,8 +112,8 @@ class LearnApp(object):
113 112 logger.debug(f'checking {qref}...')
114 113 try:
115 114 q = self.factory[qref].generate()
116   - except Exception:
117   - logger.error(f'Failed to generate "{qref}".')
  115 + except QuestionException as e:
  116 + logger.error(e)
118 117 errors += 1
119 118 continue # to next question
120 119  
... ... @@ -137,7 +136,7 @@ class LearnApp(object):
137 136 continue # to next test
138 137  
139 138 if errors > 0:
140   - logger.error(f'{errors:>6} errors found.')
  139 + logger.error(f'{errors:>6} error(s) found.')
141 140 raise LearnException('Sanity checks')
142 141 else:
143 142 logger.info(' 0 errors found.')
... ... @@ -345,7 +344,7 @@ class LearnApp(object):
345 344 logger.info(f'{m:6} topics')
346 345 logger.info(f'{q:6} answers')
347 346  
348   - # ============================================================================
  347 + # ========================================================================
349 348 # Populates a digraph.
350 349 #
351 350 # Nodes are the topic references e.g. 'my/topic'
... ... @@ -400,7 +399,7 @@ class LearnApp(object):
400 399 def make_factory(self) -> Dict[str, QFactory]:
401 400  
402 401 logger.info('Building questions factory:')
403   - factory: Dict[str, QFactory] = {}
  402 + factory = {}
404 403 g = self.deps
405 404 for tref in g.nodes():
406 405 t = g.nodes[tref]
... ... @@ -418,7 +417,7 @@ class LearnApp(object):
418 417 # within the file
419 418 for i, q in enumerate(questions):
420 419 qref = q.get('ref', str(i)) # ref or number
421   - q['ref'] = tref + ':' + qref
  420 + q['ref'] = f'{tref}:{qref}'
422 421 q['path'] = topicpath
423 422 q.setdefault('append_wrong', t['append_wrong'])
424 423  
... ...
aprendizations/questions.py
  Diff comments   View file @8f643ad
... ... @@ -72,7 +72,6 @@ class QuestionRadio(Question):
72 72 '''
73 73  
74 74 # ------------------------------------------------------------------------
75   - # FIXME marking all options right breaks
76 75 def __init__(self, q: QDict) -> None:
77 76 super().__init__(q)
78 77  
... ... @@ -86,18 +85,46 @@ class QuestionRadio(Question):
86 85 'max_tries': (n + 3) // 4 # 1 try for each 4 options
87 86 }))
88 87  
89   - # convert int to list, e.g. correct: 2 --> correct: [0,0,1,0,0]
90   - # correctness levels from 0.0 to 1.0 (no discount here!)
  88 + # check correct bounds and convert int to list,
  89 + # e.g. correct: 2 --> correct: [0,0,1,0,0]
91 90 if isinstance(self['correct'], int):
  91 + if not (0 <= self['correct'] < n):
  92 + msg = (f'Correct option not in range 0..{n-1} in '
  93 + f'"{self["ref"]}"')
  94 + raise QuestionException(msg)
  95 +
92 96 self['correct'] = [1.0 if x == self['correct'] else 0.0
93 97 for x in range(n)]
94 98  
95   - if len(self['correct']) != n:
96   - msg = ('Number of options and correct differ in '
97   - f'"{self["ref"]}", file "{self["filename"]}".')
98   - logger.error(msg)
99   - raise QuestionException(msg)
100   -
  99 + elif isinstance(self['correct'], list):
  100 + # must match number of options
  101 + if len(self['correct']) != n:
  102 + msg = (f'Incompatible sizes: {n} options vs '
  103 + f'{len(self["correct"])} correct in "{self["ref"]}"')
  104 + raise QuestionException(msg)
  105 + # make sure is a list of floats
  106 + try:
  107 + self['correct'] = [float(x) for x in self['correct']]
  108 + except (ValueError, TypeError):
  109 + msg = (f'Correct list must contain numbers [0.0, 1.0] or '
  110 + f'booleans in "{self["ref"]}"')
  111 + raise QuestionException(msg)
  112 +
  113 + # check grade boundaries
  114 + if self['discount'] and not all(0.0 <= x <= 1.0
  115 + for x in self['correct']):
  116 + msg = (f'If discount=true, correct values must be in the '
  117 + f'interval [0.0, 1.0] in "{self["ref"]}"')
  118 + raise QuestionException(msg)
  119 +
  120 + # at least one correct option
  121 + if all(x < 1.0 for x in self['correct']):
  122 + msg = (f'At least one correct option is required in '
  123 + f'"{self["ref"]}"')
  124 + raise QuestionException(msg)
  125 +
  126 + # If shuffle==false, all options are shown as defined
  127 + # otherwise, select 1 correct and choose a few wrong ones
101 128 if self['shuffle']:
102 129 # lists with indices of right and wrong options
103 130 right = [i for i in range(n) if self['correct'][i] >= 1]
... ... @@ -123,7 +150,7 @@ class QuestionRadio(Question):
123 150 # final shuffle of the options
124 151 perm = random.sample(range(self['choose']), k=self['choose'])
125 152 self['options'] = [str(options[i]) for i in perm]
126   - self['correct'] = [float(correct[i]) for i in perm]
  153 + self['correct'] = [correct[i] for i in perm]
127 154  
128 155 # ------------------------------------------------------------------------
129 156 # can assign negative grades for wrong answers
... ... @@ -131,10 +158,13 @@ class QuestionRadio(Question):
131 158 super().correct()
132 159  
133 160 if self['answer'] is not None:
134   - x = self['correct'][int(self['answer'])]
135   - if self['discount']:
136   - n = len(self['options']) # number of options
137   - x_aver = sum(self['correct']) / n
  161 + x = self['correct'][int(self['answer'])] # get grade of the answer
  162 + n = len(self['options'])
  163 + x_aver = sum(self['correct']) / n # expected value of grade
  164 +
  165 + # note: there are no numerical errors when summing 1.0s so the
  166 + # x_aver can be exactly 1.0 if all options are right
  167 + if self['discount'] and x_aver != 1.0:
138 168 x = (x - x_aver) / (1.0 - x_aver)
139 169 self['grade'] = x
140 170  
... ... @@ -168,12 +198,32 @@ class QuestionCheckbox(Question):
168 198 'max_tries': max(1, min(n - 1, 3))
169 199 }))
170 200  
  201 + # must be a list of numbers
  202 + if not isinstance(self['correct'], list):
  203 + msg = 'Correct must be a list of numbers or booleans'
  204 + raise QuestionException(msg)
  205 +
  206 + # must match number of options
171 207 if len(self['correct']) != n:
172   - msg = (f'Options and correct size mismatch in '
173   - f'"{self["ref"]}", file "{self["filename"]}".')
174   - logger.error(msg)
  208 + msg = (f'Incompatible sizes: {n} options vs '
  209 + f'{len(self["correct"])} correct in "{self["ref"]}"')
175 210 raise QuestionException(msg)
176 211  
  212 + # make sure is a list of floats
  213 + try:
  214 + self['correct'] = [float(x) for x in self['correct']]
  215 + except (ValueError, TypeError):
  216 + msg = (f'Correct list must contain numbers or '
  217 + f'booleans in "{self["ref"]}"')
  218 + raise QuestionException(msg)
  219 +
  220 + # check grade boundaries (FUTURE)
  221 + # if self['discount'] and not all(0.0 <= x <= 1.0
  222 + # for x in self['correct']):
  223 + # msg = (f'If discount=true, correct values must be in the '
  224 + # f'interval [0.0, 1.0] in "{self["ref"]}"')
  225 + # raise QuestionException(msg)
  226 +
177 227 # if an option is a list of (right, wrong), pick one
178 228 options = []
179 229 correct = []
... ... @@ -526,17 +576,21 @@ class QFactory(object):
526 576 stdin=q['stdin'])
527 577 q.update(out)
528 578  
529   - # Finally we create an instance of Question()
  579 + # Get class for this question type
530 580 try:
531   - qinstance = self._types[q['type']](QDict(q)) # of matching class
532   - except QuestionException as e:
533   - logger.error(e)
534   - raise e
  581 + qclass = self._types[q['type']]
535 582 except KeyError:
536 583 logger.error(f'Invalid type "{q["type"]}" in "{q["ref"]}"')
537 584 raise
538   - else:
539   - return qinstance
  585 +
  586 + # Finally create an instance of Question()
  587 + try:
  588 + qinstance = qclass(QDict(q))
  589 + except QuestionException as e:
  590 + # logger.error(e)
  591 + raise e
  592 +
  593 + return qinstance
540 594  
541 595 # ------------------------------------------------------------------------
542 596 def generate(self) -> Question:
... ...
aprendizations/serve.py
  Diff comments   View file @8f643ad
... ... @@ -254,7 +254,6 @@ class TopicHandler(BaseHandler):
254 254 class FileHandler(BaseHandler):
255 255 @tornado.web.authenticated
256 256 async def get(self, filename):
257   - # logger.debug(f'[FileHandler] {filename}')
258 257 uid = self.current_user
259 258 public_dir = self.learn.get_current_public_dir(uid)
260 259 filepath = path.expanduser(path.join(public_dir, filename))
... ... @@ -331,7 +330,7 @@ class QuestionHandler(BaseHandler):
331 330 user = self.current_user
332 331 answer = self.get_body_arguments('answer') # list
333 332 qid = self.get_body_arguments('qid')[0]
334   - logger.debug(f'[QuestionHandler] user={user}, answer={answer}')
  333 + logger.debug(f'[QuestionHandler] answer={answer}')
335 334  
336 335 # --- check if browser opened different questions simultaneously
337 336 if qid != self.learn.get_current_question_id(user):
... ...
config/logger-debug.yaml
  Diff comments   View file @8f643ad
... ... @@ -2,50 +2,52 @@
2 2 version: 1
3 3  
4 4 formatters:
5   - void:
6   - format: ''
7   - standard:
8   - format: '%(asctime)s | %(thread)-15d | %(levelname)-8s | %(module)-10s | %(funcName)-20s | %(message)s'
9   - # datefmt: '%H:%M:%S'
  5 + void:
  6 + format: ''
  7 + standard:
  8 + format: '%(asctime)s | %(levelname)-8s | %(module)-10s | %(funcName)-22s |
  9 + %(message)s'
  10 + # | %(thread)-15d
  11 + # datefmt: '%H:%M:%S'
10 12  
11 13 handlers:
12   - default:
13   - level: 'DEBUG'
14   - class: 'logging.StreamHandler'
15   - formatter: 'standard'
16   - stream: 'ext://sys.stdout'
  14 + default:
  15 + level: 'DEBUG'
  16 + class: 'logging.StreamHandler'
  17 + formatter: 'standard'
  18 + stream: 'ext://sys.stdout'
17 19  
18 20 loggers:
19   - '':
20   - handlers: ['default']
21   - level: 'DEBUG'
22   -
23   - 'aprendizations.factory':
24   - handlers: ['default']
25   - level: 'DEBUG'
26   - propagate: false
27   -
28   - 'aprendizations.student':
29   - handlers: ['default']
30   - level: 'DEBUG'
31   - propagate: false
32   -
33   - 'aprendizations.learnapp':
34   - handlers: ['default']
35   - level: 'DEBUG'
36   - propagate: false
37   -
38   - 'aprendizations.questions':
39   - handlers: ['default']
40   - level: 'DEBUG'
41   - propagate: false
42   -
43   - 'aprendizations.tools':
44   - handlers: ['default']
45   - level: 'DEBUG'
46   - propagate: false
47   -
48   - 'aprendizations.serve':
49   - handlers: ['default']
50   - level: 'DEBUG'
51   - propagate: false
  21 + '':
  22 + handlers: ['default']
  23 + level: 'DEBUG'
  24 +
  25 + 'aprendizations.factory':
  26 + handlers: ['default']
  27 + level: 'DEBUG'
  28 + propagate: false
  29 +
  30 + 'aprendizations.student':
  31 + handlers: ['default']
  32 + level: 'DEBUG'
  33 + propagate: false
  34 +
  35 + 'aprendizations.learnapp':
  36 + handlers: ['default']
  37 + level: 'DEBUG'
  38 + propagate: false
  39 +
  40 + 'aprendizations.questions':
  41 + handlers: ['default']
  42 + level: 'DEBUG'
  43 + propagate: false
  44 +
  45 + 'aprendizations.tools':
  46 + handlers: ['default']
  47 + level: 'DEBUG'
  48 + propagate: false
  49 +
  50 + 'aprendizations.serve':
  51 + handlers: ['default']
  52 + level: 'DEBUG'
  53 + propagate: false
... ...
demo/astronomy/solar-system/questions.yaml
  Diff comments   View file @8f643ad
... ... @@ -28,6 +28,7 @@
28 28 - Têm todos o mesmo tamanho
29 29 # opcional
30 30 correct: 2
  31 + # discount: true
31 32 shuffle: false
32 33 solution: |
33 34 O maior planeta é Júpiter. Tem uma massa 1000 vezes inferior ao Sol, mas
... ...
demo/math/addition/questions.yaml
  Diff comments   View file @8f643ad
... ... @@ -5,6 +5,7 @@
5 5 script: addition-two-digits.py
6 6 args: [10, 20]
7 7  
  8 +# ---------------------------------------------------------------------------
8 9 - type: checkbox
9 10 ref: addition-properties
10 11 title: Propriedades da adição
... ...