Merge branch 'dev'

Miguel Barão
2 parents f6fdbdbd a0ab1672
Showing 30 changed files with 1174 additions and 1520 deletions Show diff stats
BUGS.md
README.md
demo/demo.yaml
demo/questions/generate-question.py
demo/questions/generators/generate-question.py
demo/questions/questions-tutorial.yaml
mypy.ini
package.json
perguntations/__init__.py
perguntations/app.py
perguntations/initdb.py
perguntations/main.py
perguntations/models.py
perguntations/parser_markdown.py
perguntations/questions.py
perguntations/serve.py
perguntations/static/js/admin.js
perguntations/templates/admin.html
perguntations/templates/grade.html
perguntations/templates/question-information.html
  
 # BUGS
  
-- show_points nao esta a funcionar (mostra sempre)
-- esta a corrigir código JOBE mesmo que nao tenha respondido???
-- correct devia poder ser corrido mais que uma vez (por exemplo para alterar
-  cotacoes, corrigir perguntas)
-- nao esta a mostrar imagens?? internal server error?
-- guardar testes em JSON assim que sao atribuidos aos alunos (ou guardados
-  inicialmente com um certo nome, e atribuidos posteriormente ao aluno).
-- cookies existe um `perguntations_user` e um user. De onde vem o user?
-- QuestionCode falta reportar nos comments os vários erros que podem ocorrer
-  (timeout, etc)
-- algumas vezes a base de dados guarda o mesmo teste em duplicado. ver se dois
-  submits dao origem a duas correcções.  talvez a base de dados devesse ter
-  como chave do teste um id que fosse único desse teste particular (não um auto
-  counter, nem ref do teste)
-- em caso de timeout na submissão (e.g. JOBE ou script nao responde) a
-  correcção não termina e o teste não é guardado.
+- correct devia poder ser corrido mais que uma vez (por exemplo para alterar cotacoes, corrigir perguntas)
+- guardar testes em JSON assim que sao atribuidos aos alunos (ou guardados inicialmente com um certo nome, e atribuidos posteriormente ao aluno).
+- cookies existe um perguntations_user e um user. De onde vem o user?
+- QuestionCode falta reportar nos comments os vários erros que podem ocorrer (timeout, etc)
+- algumas vezes a base de dados guarda o mesmo teste em duplicado. ver se dois submits dao origem a duas correcções.
+talvez a base de dados devesse ter como chave do teste um id que fosse único desse teste particular (não um auto counter, nem ref do teste)
+- em caso de timeout na submissão (e.g. JOBE ou script nao responde) a correcção não termina e o teste não é guardado.
 - grade gives internal server error??
 - reload do teste recomeça a contagem no inicio do tempo.
-- em admin, quando `scale_max` não é 20, as cores das barras continuam a
-  reflectir a escala 0,20. a tabela teste na DB não tem a escala desse teste.
-- em grade.html as barras estao normalizadas para os limites `scale_min` e max
-  do teste actual e nao dos testes realizados no passado (tabela test devia
-      guardar a escala).
-- codigo `hello   world` nao esta a preservar o whitespace. O renderer de
-  markdown gera a tag `<code>` que não preserva whitespace. Necessario
-  adicionar `<pre>.`
-- mensagems de erro do assembler aparecem na mesma linha na correcao e nao
-  fazerm rendering do `$t`, ver se servidor faz parse do markdown dessas
-  mensagens.
+- em admin, quando scale_max não é 20, as cores das barras continuam a reflectir a escala 0,20. a tabela teste na DB não tem a escala desse teste.
+- em grade.html as barras estao normalizadas para os limites scale_min e max do teste actual e nao dos testes realizados no passado (tabela test devia guardar a escala).
+- codigo `hello   world` nao esta a preservar o whitespace. O renderer de markdown gera a tag <code> que não preserva whitespace. Necessario adicionar <pre>.
+- mensagems de erro do assembler aparecem na mesma linha na correcao e nao fazerm rendering do `$t`, ver se servidor faz parse do markdown dessas mensagens.
 - a revisao do teste não mostra as imagens.
-- `Test.reset_answers()` unused.
+- Test.reset_answers() unused.
 - teste nao esta a mostrar imagens de vez em quando.???
 - show-ref nao esta a funcionar na correccao (pelo menos)
  
 # TODO
  
-- permitir que textarea nao tenha correct definido, da sempre 0 e é para
-  corrigir manualmente.
-- permitir corrigir mais que uma vez, actualizando a base de dados.
-- yaml schemas para validar os ficheiros yaml (ver cerberus e jsonschema)
+- JOBE correct async
+- esta a corrigir código JOBE mesmo que nao tenha respondido???
 - permitir remover alunos que estão online para poderem comecar de novo.
-- guardar nota final grade truncado em zero e sem ser truncado (quando é
-  necessário fazer correcções à mão às perguntas, é necessário o valor não
-  truncado)
-- stress tests. use [locust](https://locust.io)
+- guardar nota final grade truncado em zero e sem ser truncado (quando é necessário fazer correcções à mão às perguntas, é necessário o valor não truncado)
+- stress tests. use https://locust.io
 - wait for admin to start test. (students can be allowed earlier)
-- impedir os eventos copy/paste. alunos usam isso para trazer codigo ja feito
-  nos computadores. Obrigar a fazer reset? fazer um copy automaticamente?
-- na pagina admin, mostrar com cor vermelha as horas de entrada dos alunos que
-  ja tenham excedido o tempo
-- retornar None quando nao ha alteracoes relativamente à última vez.  ou usar
-  push (websockets?)
-- mudar ref do test para `test_id` (ref já é usado nas perguntas)
+- impedir os eventos copy/paste. alunos usam isso para trazer codigo ja feito nos computadores. Obrigar a fazer reset? fazer um copy automaticamente?
+- na pagina admin, mostrar com cor vermelha as horas de entrada dos alunos que ja tenham excedido o tempo
+- retornar None quando nao ha alteracoes relativamente à última vez.
+ou usar push (websockets?)
+- mudar ref do test para test_id (ref já é usado nas perguntas)
 - servidor ntpd no x220 para configurar a data/hora dos portateis dell
-- sala de espera: autorização dada, mas teste não disponível até que seja dada
-  ordem para começar.
-- alunos com necessidades especiais nao podem ter autosubmit. ter um
-  `autosubmit_exceptions`: ['123', '456']
+- sala de espera: autorização dada, mas teste não disponível até que seja dada ordem para começar.
+- alunos com necessidades especiais nao podem ter autosubmit. ter um autosubmit_exceptions: ['123', '456']
 - submissao fazer um post ajax?
 - adicionar opcao para eliminar um teste em curso.
 - enviar resposta de cada pergunta individualmente.
-- experimentar gerador de svg que inclua no markdown da pergunta e ver se
-  funciona.
-- quando ha varias perguntas para escolher, escolher sucessivamente em vez de
-  aleatoriamente.
+- experimentar gerador de svg que inclua no markdown da pergunta e ver se funciona.
+- quando ha varias perguntas para escolher, escolher sucessivamente em vez de aleatoriamente.
 - como refrescar a tabela de admin sem fazer reload da pagina?
-- botao "testar resposta" que valida codigo relativamente a syntax, mas nao
-  classifica. perguntas devem ter opcao validate: script.py. Aluno pressiona
-  botao e codigo é enviado para servidor para validação, feedback é mostrado na
-  pagina de teste.
-- test: botao submeter valida se esta online com um post `willing_to_submit`,
-  se estiver online, mostra mensagem de confirmacao, caso contrario avisa que
-  nao esta online.
+- botao "testar resposta" que valida codigo relativamente a syntax, mas nao classifica. perguntas devem ter opcao validate: script.py. Aluno pressiona botao e codigo é enviado para servidor para validação, feedback é mostrado na pagina de teste.
+- test: botao submeter valida se esta online com um post willing_to_submit, se estiver online, mostra mensagem de confirmacao, caso contrario avisa que nao esta online.
 - test: Cada pergunta respondida é logo submetida.
 - test: calculadora javascript.
 - admin: histograma das notas.
 - admin: mostrar as horas a que o teste terminou para os testes terminados.
 - admin: histograma das notas.
 - admin: mostrar teste gerado para aluno (tipo review).
-- fazer renderer para formulas com mathjax serverside (mathjax-node) ou usar
-  katex.
+- fazer renderer para formulas com mathjax serverside (mathjax-node) ou usar katex.
 - fazer renderer para imagens, com links /file?ref=xpto;name=zzz.jpg
 - fazer renderer para linguagem assembly mips?
-- cancelar teste no menu admin. Dado o numero de aluno remove teste e faz
-  logout do aluno.
+- cancelar teste no menu admin. Dado o numero de aluno remove teste e faz logout do aluno.
+- mathjax-node:
+    sudo pkg install node npm
+    npm install mathjax-node mathjax-node-cli    # pacotes em ~/node_modules
+    node_modules/mathjax-node-cli/bin/tex2svg '\sqrt{x}'
+  usar isto para gerar svg que passa a fazer parte do texto da pergunta (markdown suporta tags svg?)
+  fazer funçao tex() que recebe formula e converte para svg. exemplo:
+  fr'''A formula é {tex("\sqrt{x]}")}'''
 - Gerar pdf's com todos os testes no final (pdfkit).
-- manter registo dos unfocus durante o teste e de qual a pergunta visivel nesse
-  momento
+- manter registo dos unfocus durante o teste e de qual a pergunta visivel nesse momento
 - permitir varios testes, aluno escolhe qual o teste que quer fazer.
 - se ocorrer um erro na correcçao avisar aluno para contactar o professor.
-- abrir o teste numa janela maximizada e que nao permite que o aluno a
-  redimensione/mova?
-- detectar scroll e enviar posição para servidor (analise de scroll para
-  detectar copianço? ou simplesmente para analisar como os alunos percorrem o
-  teste)
-- aviso na pagina principal para quem usa browser incompativel ou settings
-  esquisitos... Apos login pode ser enviado e submetido um exemplo de teste
-  para verificar se browser consegue submeter? ha alunos com javascript
-  bloqueado?
+- abrir o teste numa janela maximizada e que nao permite que o aluno a redimensione/mova?
+- detectar scroll e enviar posição para servidor (analise de scroll para detectar copianço? ou simplesmente para analisar como os alunos percorrem o teste)
+- aviso na pagina principal para quem usa browser incompativel ou settings esquisitos... Apos login pode ser enviado e submetido um exemplo de teste para verificar se browser consegue submeter? ha alunos com javascript bloqueado?
 - criar perguntas de outros tipos, e.g. associação, ordenação, varios textinput
-- perguntas para professor corrigir mais tarde. permitir que review possa
-  alterar as notas
+- perguntas para professor corrigir mais tarde. permitir que review possa alterar as notas
 - fazer uma calculadora javascript e por no menu. surge como modal
  
 # FIXED
  
-- JOBE correct async
 - testar as perguntas todas no início do teste como o aprendizations.
-- adicionar identificacao do aluno no jumbotron inicial do teste, para que ao
-  imprimir para pdf a identificacao do aluno venha escrita no documento.
+- adicionar identificacao do aluno no jumbotron inicial do teste, para que ao imprimir para pdf a identificacao do aluno venha escrita no documento.
 - internal server error quando em --review, download csv detalhado.
-- perguntas repetidas (mesma ref) dao asneira, porque a referencia é usada como
-  chave em varios sitios e as chaves nao podem ser dupplicadas.  da asneira
-  pelo menos na funcao `get_questions_csv`. na base de dados tem de estar
-  registado tb o numero da pergunta, caso contrario é impossível saber a qual
-  corresponde.
+- perguntas repetidas (mesma ref) dao asneira, porque a referencia é usada como chave em varios sitios e as chaves nao podem ser dupplicadas.
+  da asneira pelo menos na funcao get_questions_csv. na base de dados tem de estar registado tb o numero da pergunta, caso contrario é impossível saber a qual corresponde.
 - mostrar unfocus e window area em /admin
-- CRITICAL se answer for `i<n` a revisão de provas mostra apenas i (interpreta
-  `<` como tag?)
+- CRITICAL se answer for `i<n` a revisão de provas mostra apenas i (interpreta `<` como tag?)
 - botao de autorizar desliga-se, fazer debounce.
 - link na pagina com a nota para voltar ao principio.
 - default logger config mostrar horas com segundos
@@ -128,45 +88,33 @@
 - servidor nao esta a lidar com eventos resize.
 - sock.bind(sockaddr)  OSError: [Errno 48] Address already in use
 - dizer quanto desconta em cada pergunta de escolha multipla
-- se houver erros a abrir ficheiros .yaml de perguntas, depois dos testes diz
-  "No errors found".
-- se faltarem files na especificação do teste, o check não detecta e factory
-  não gera para essas perguntas.
+- se houver erros a abrir ficheiros .yaml de perguntas, depois dos testes diz "No errors found".
+- se faltarem files na especificação do teste, o check não detecta e factory não gera para essas perguntas.
 - nao esta a usar points das perguntas
 - quando se clica no texto de uma opcao, salta para outro lado na pagina.
-- suportar cotacao to teste diferente de 20 (e.g. para juntar perguntas em
-  papel). opcao "points: 18" que normaliza total para 18 em vez de 20.
+- suportar cotacao to teste diferente de 20 (e.g. para juntar perguntas em papel). opcao "points: 18" que normaliza total para 18 em vez de 20.
 - fazer package para instalar perguntations com pip.
 - pymips: nao pode executar syscalls do spim.
 - exception sqlalchemy relacionada com threads.
 - acrescentar logger.conf que sirva de base.
-- questions.py textarea has a abspath which does not make sense! why is it
-  there? not working for perguntations, but seems to work for aprendizations
-- textarea foi modificado em aprendizations para receber cmd line args.
-  corrigir aqui tb.
+- questions.py textarea has a abspath which does not make sense! why is it there? not working for perguntations, but seems to work for aprendizations
+- textarea foi modificado em aprendizations para receber cmd line args. corrigir aqui tb.
 - usar npm para instalar javascript
 - se aluno entrar com l12345 rebenta. numa funcao get_... ver imagem no ipad
-- no test3 está contar 1.0 valores numa pergunta do tipo info? acontece para
-  type: info, e não para type: information
-- default correct in checkbox must be 1.0, so that the pairs (right,wrong)
-  still work with `correct` left undefined.
+- no test3 está contar 1.0 valores numa pergunta do tipo info? acontece para type: info, e não para type: information
+- default correct in checkbox must be 1.0, so that the pairs (right,wrong) still work with `correct` left undefined.
 - textarea com codemirror
 - decorador para user 0, evita o "if uid==0" em muitas funcoes.
-- numeric interval deve converter respostas que usam virgulas para pontos
-  decimais
-- `self.testapp.get_json_filename_of_test(test_id)` retorna None quando test_id
-  nao existe.
-- o eventloop está a bloquear. correção do teste é blocking. usar
-  threadpoolexecutor?
-- substituir get_event_loop por get_runnint_loop
+- numeric interval deve converter respostas que usam virgulas para pontos decimais
+- self.testapp.get_json_filename_of_test(test_id) retorna None quando test_id nao existe.
+- o eventloop está a bloquear. correção do teste é blocking. usar threadpoolexecutor?
+- substituir get_event_loop por get_runnint_loop (ver https://docs.python.org/3/library/asyncio-eventloop.html)
 - review nao esta a funcionar
-- servir imagens das perguntas não funciona. Necessario passar a ref da
-  pergunta no link para poder ajustar o path no FileHandler.
+- servir imagens das perguntas não funciona. Necessario passar a ref da pergunta no link para poder ajustar o path no FileHandler.
 - a primeira coluna da tabela admin deveria estar sempre ordenada.
 - abortar depois de testar todas as perguntas, caso haja algum erro.
 - imagens jpg/png nas perguntas.
-- initdb está a inicializar com passwords iguais aos numeros. deveria ser vazio
-  para alunos definirem.
+- initdb está a inicializar com passwords iguais aos numeros. deveria ser vazio para alunos definirem.
 - upgrade popper e fazer link.
 - mover scripts js para head, com defer. ver todos os templates.
 - update fontawesome to 5.
@@ -176,8 +124,7 @@
 - md_to_html() nao usa o segundo argumento q. pode retirar-se dos templates?
 - config/logger.yaml ainda é do cherrypy...
 - uniformizar question.py com a de aprendizations...
-- qual a diferenca entre md_to_html e md_to_html_review, parece desnecessario
-  haver dois.
+- qual a diferenca entre md_to_html e md_to_html_review, parece desnecessario haver dois.
 - converter markdown para mistune.
 - como alterar configuracao para mostrar logs de debug?
 - espaco no final das tabelas.
@@ -193,43 +140,31 @@
 - text-numeric não está a gerar a pergunta. faltam templates?
 - testar perguntas warning/warn
 - qd user 0 faz logout rebenta.
-- Quando grava JSON do teste deve usar 'path' tal como definido na configuração
-  e não expandido. Isto porque em OSX /home é /Users e quando se muda de um
-  sistema para outro não encontra os testes. Assim, usando ~ na configuração
-  deveria funcionar sempre.
-- configuração do teste não joga bem com o do aprendizations. Em particular os
-  scripts não ficam com o mesmo path!!!
+- Quando grava JSON do teste deve usar 'path' tal como definido na configuração e não expandido. Isto porque em OSX /home é /Users e quando se muda de um sistema para outro não encontra os testes. Assim, usando ~ na configuração deveria funcionar sempre.
+- configuração do teste não joga bem com o do aprendizations. Em particular os scripts não ficam com o mesmo path!!!
 - configurar pf em freebsd, port forward 80 -> 8080. documentacao
 - barras com notas em grade estão desalinhadas.
 - erros nos generators devem ser ERROR e não WARNING.
-- se directorio "logs" não existir no directorio actual aborta com mensagem de
-  erro.
-- se um teste tiver a mesma pergunta repetida (ref igual), rebenta na
-  correcçao. As respostas são agregadas numa lista para cada ref. Ex: {'ref1':
-  'resposta1', 'ref2': ['resposta2a', 'resposta2b']}
-- usar [](http://fontawesome.io/examples/) em vez dos do bootstrap3
+- se directorio "logs" não existir no directorio actual aborta com mensagem de erro.
+- se um teste tiver a mesma pergunta repetida (ref igual), rebenta na correcçao. As respostas são agregadas numa lista para cada ref. Ex: {'ref1': 'resposta1', 'ref2': ['resposta2a', 'resposta2b']}
+- usar http://fontawesome.io/examples/ em vez dos do bootstrap3
 - se pergunta tiver 'type:' errado, rebenta.
 - se submeter um teste so com information, da divisao por zero.
-- se save_answers nao existir, da warning que nao serao gravados, mas sao
-  sempre gravados! pagina de administracao diz --not being saved--
+- se save_answers nao existir, da warning que nao serao gravados, mas sao sempre gravados! pagina de administracao diz --not being saved--
 - first login é INFO e não WARNING
 - /review não mostra imagens porque precisa que teste esteja a decorrer...
 - visualizar um teste ja realizado na página de administração
-- Depois da correcção, mostra testes realizados que não foram realizados pelo
-  próprio
-- detectar se janela perde focus e alertar o prof
-- server nao esta a receber eventos focus/blur dos utilizadores diferentes de
-  '0', estranho...
+- Depois da correcção, mostra testes realizados que não foram realizados pelo próprio
+- detectar se janela perde focus e alertar o prof (http://stackoverflow.com/questions/1060008/is-there-a-way-to-detect-if-a-browser-window-is-not-currently-active)
+- server nao esta a receber eventos focus/blur dos utilizadores diferentes de '0', estranho...
 - permitir adicionar imagens nas perguntas.
 - detect_unfocus.js so funciona se estiver inline no html. porquê???
 - inserir novo aluno /admin não fecha.
 - se aluno desistir, ainda fica marcado como online
 - give dá None em vez de 0.0
 - debug mode: log levels not working
-- Se aluno fizer logout, o teste não é gravado e ficamos sem registo do teste
-  que o aluno viu.
-- criar sqlalchemy sessions dentro de app de modo a estarem associadas a
-  requests. ver se é facil usar with db:(...) para criar e fechar sessão.
+- Se aluno fizer logout, o teste não é gravado e ficamos sem registo do teste que o aluno viu.
+- criar sqlalchemy sessions dentro de app de modo a estarem associadas a requests. ver se é facil usar with db:(...) para criar e fechar sessão.
 - sqlalchemy queixa-se de threads.
 - SQLAlchemy em vez da classe database.
 - replace sys.exit calls
@@ -238,21 +173,20 @@
 - configuracao dos logs cherrypy para se darem bem com os outros
 - browser e ip usados gravado no test.
 - botões allow all/deny all.
-- mostrar botão de reset apenas no final da pagina, com edit para escrever o
-  número.
-- aluno faz login, mas fecha browser, ficando no estado (online,deny). Ao
-  tentar login com outro browser está deny e o prof não consegue pô-lo em allow
-  pois já não está na lista. => solucao é manter todos os alunos numa tabela.
-- pagina de login nao esta a apresentar bem. parece que precisa de autorizacao
-  para aceder a /static...
+- mostrar botão de reset apenas no final da pagina, com edit para escrever o número.
+- aluno faz login, mas fecha browser, ficando no estado (online,deny). Ao tentar login com outro browser está deny e o prof não consegue pô-lo em allow pois já não está na lista. => solucao é manter todos os alunos numa tabela.
+- pagina de login nao esta a apresentar bem. parece que precisa de autorizacao para aceder a /static...
 - Não mostrar Professor nos activos em /admin
 - /admin mostrar actualizações automaticamente?
 - se no teste uma das "ref" nao existir nos ficheiros de perguntas, rebenta.
-- alunos podem estar online, mas browser perder sessao => nao conseguem mais
-  entrar porque a App pensa que estão online. Permitir login e dar o mesmo
-  teste.
-- pagina de management dos alunos.  mostrar online ordenados por hora de login,
-  offline por número.  permitir reset da pw e allow/disallow
+- alunos podem estar online, mas browser perder sessao => nao conseguem mais entrar porque a App pensa que estão online. Permitir login e dar o mesmo teste.
+- pagina de management dos alunos.
+    mostrar online ordenados por hora de login, offline por número.
+    permitir reset da pw e allow/disallow
 - script de correcção pode enviar dicionario yaml com grade e comentarios. ex:
-  grade: 0.5 comments: Falhou na função xpto.  os comentários são guardados no
-  teste (ficheiro) ou enviados para o browser no modo practice.
+    grade: 0.5
+    comments: Falhou na função xpto.
+    os comentários são guardados no teste (ficheiro) ou enviados para o browser no modo practice.
+- testar regex na definicao das perguntas. como se faz rawstring em yaml?
+    singlequote? problemas de backslash??? sim... necessário fazer \\ em varios casos, mas não é claro! e.g. \n é convertido em espaço mas \w é convertido em \\ e w. Solução (http://stackoverflow.com/questions/10771163/python-interpreting-a-regex-from-a-yaml-config-file) é fazer
+            correct: !regex '^(yes|no)'
@@ -11,26 +11,27 @@
  
 ## Requirements
  
-The webserver is a python application that requires `>=python3.7` and the package installer for python `pip`. The node package management `npm` is also necessary in order to install the javascript libraries.
+The webserver is a python application that requires `>=python3.8` and the
+package installer for python `pip`. The node package management `npm` is also
+necessary in order to install the javascript libraries.
  
 ```sh
 sudo apt install python3 python3-pip npm                     # Ubuntu
-sudo pkg install python37 py37-sqlite3 py37-pip npm          # FreeBSD
-sudo port install python37 py37-pip py37-setuptools npm6     # MacOS (macports)
+sudo pkg install python38 py38-sqlite3 py38-pip npm          # FreeBSD
+sudo port install python38 py38-pip py38-setuptools npm6     # MacOS (macports)
 ```
  
-To make the `pip` install packages to a local directory, the file `pip.conf` should be configured as follows:
+To make the `pip` install packages to a local directory, the file `pip.conf`
+should be configured as follows:
  
 ```ini
 [global]
 user=yes
-
-[list]
-format=columns
 ```
  
 This file is usually in `~/.config/pip/` in Linux and FreeBSD. In MacOS it's in
-`~/Library/Application Support/pip/`. You may have to create it, if it doesn't exist yet.
+`~/Library/Application Support/pip/`. You may have to create it, if it doesn't
+exist yet.
  
 ---
  
@@ -42,19 +43,21 @@ Download and install:
 git clone https://git.xdi.uevora.pt/mjsb/perguntations.git
 cd perguntations
 npm install
-pip3 install .
+pip install .
 ```
  
-The command `npm` installs the javascript libraries and then `pip3` installs the python webserver. This will also install any required dependencies.
+The command `npm` installs the javascript libraries and then `pip` installs the
+python webserver. This will also install any required dependencies.
  
-**Atention!** `pip3 install .` must run **after** `npm install`, otherwise the javascript libraries will not be found during the install.
+**Atention!** `pip install .` must be run **after** `npm install`, otherwise
+the javascript libraries will not be found during the install.
  
 ---
  
 ## Setup
  
-The server will run a `https` server and requires valid certificates.
-There are two possibilities to generate the certificates:
+The server will run a `https` server and requires valid certificates.  There
+are two possibilities to generate the certificates:
  
 - public server with static IP address and registered domain name;
 - private server on a local network isolated from the internet.
@@ -74,38 +77,35 @@ cd ~/.local/share/certs
 openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes
 ```
  
-To have a true certificate, like the ones issued by LetsEncrypt, the host
-must have a registered domain name.
-See instructions for FreeBSD at the end of this page.
+To have a true certificate, like the ones issued by LetsEncrypt, the host must
+have a registered domain name.  See instructions for FreeBSD at the end of this
+page.
  
 ---
  
 ## Running a demo
  
-The directory `demo` in the repository includes a demo test that can be used
-as a template for your own tests and questions.
-
-To run the demonstration test you need to initialize the database using one of
-the following methods:
+The directory `demo` in the repository includes a demo test that can be used as
+a template for your own tests and questions.  To run the demonstration test you
+need to initialize the database using one of the following methods:
  
 ```sh
 cd perguntations/demo
-
 initdb students.csv                 # initialize from a CSV file
 initdb --admin                      # only adds the administrator account
 initdb --add 123 "Asterix Gaules"   # add one student
 ```
  
 This will create or update a `students.db` file that contains a sqlite3
-database.
-The database stores user passwords and grades, but not the actual tests.
+database.  The database stores user passwords and grades, but not the actual
+tests.
  
-A test is specified in a single `yaml` file.
-The demo already includes the `demo.yaml` that you can play with.
+A test is specified in a single `yaml` file.  The file `demo.yaml` is a test
+specification that you can play with.
  
-The complete tests submitted by the students are stored in JSON files in the
-directory defined in `demo.yaml` under the option `answers_dir: ans`.
-We also have to create this directory manually:
+The answered tests submitted by the students are stored in JSON files in a
+directory defined in `demo.yaml` under the option `answers_dir: ans`.  We also
+have to create this directory manually:
  
 ```sh
 mkdir ans                           # directory where the tests will be saved
@@ -114,38 +114,36 @@ mkdir ans                           # directory where the tests will be saved
 Start the server and run the `demo.yaml` test:
  
 ```sh
-perguntations demo.yaml         # run demo test
+perguntations demo.yaml             # run demo test
 ```
  
 Several options are available, run `perguntations --help` for a list.
  
-The server listens on port 8443 of all IPs of all network interfaces.
-Open the browser at `http://127.0.0.1:8443/` and login as user number `0`
+The server listens on port 8443 of all IPs of all network interfaces.  Open the
+browser at `http://127.0.0.1:8443/` and login as user number `0`
 (administrator) and choose any password you want. The password is defined on
 first login.
+
 After logging in, you will be redirected to the administration page that shows
 all the students and their current state.
  
 1. Authorize students by clicking the checkboxes.
 2. Open a different browser (or exit administrator) at `http://127.0.0.1:8443/`
-and login as one of the authorized students.
-Answer the questions and submit. You should get a grade at the end.
+   and login as one of the authorized students.  Answer the questions and
+   submit. You should get a grade at the end.
  
-The server can be stoped from the terminal with `^C`.
+The server can be stoped from the terminal with `^C` (Control+C).
  
 ---
  
 ## Running on lower ports
  
 Ports 80 and 443 are reserved for the root user and and this software **should
-NOT be run as root**.
-Instead, tcp traffic can be forwarded from port 443 to 8443 where the server is
-listening.
-The details depend on the operating system/firewall.
-
-### debian
+NOT be run as root**.  Instead, tcp traffic can be forwarded from port 443 to
+8443 where the server is listening.  The details depend on the operating
+system/firewall.
  
-FIXME: Untested
+### debian (FIXME: Untested)
  
 ```.sh
 iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-ports 8443
@@ -166,25 +164,29 @@ Explanation:
  
 Edit `/etc/pf.conf`:
  
-    ext_if="wlan1"
-    rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
-    rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
+```conf
+ext_if="wlan1"
+rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
+rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
+```
  
-The `wlan1` should be the name of the network interface.
-Use `ext_if="vtnet0"` for guest additions under virtual box.
+The `wlan1` should be the name of the network interface.  Use `ext_if="vtnet0"`
+for guest additions under virtual box.
  
 Start firewall with `sudo service pf start`.
  
 Optionally, to activate pf on boot, edit `rc.conf`:
  
-    pf_enable="YES"
-    pf_flags=""
-    pf_rules="/etc/pf.conf"
+```conf
+pf_enable="YES"
+pf_flags=""
+pf_rules="/etc/pf.conf"
  
-    # optional logging:
-    pflog_enable="YES"
-    pflog_flags=""
-    pflog_logfile="/var/log/pflog"
+# optional logging:
+pflog_enable="YES"
+pflog_flags=""
+pflog_logfile="/var/log/pflog"
+```
  
 ## Generating certificates with LetsEncript (FreeBSD)
  
@@ -192,7 +194,7 @@ Generating certificates for a public server (FreeBSD) requires a registered
 domain with fixed IP.
  
 ```sh
-sudo pkg install py27-certbot                   # FreeBSD
+sudo pkg install py38-certbot                   # FreeBSD 13
 sudo service pf stop                            # disable pf firewall (FreeBSD)
 sudo certbot certonly --standalone -d www.example.com
 sudo service pf start                           # enable pf firewall
@@ -208,47 +210,52 @@ chmod 400 cert.pem privkey.pem
 Certificate renewals can be done as follows:
  
 ```sh
-sudo service pf stop   # shutdown firewall
+sudo service pf stop        # shutdown firewall
 sudo certbot renew
-sudo service pf start  # start firewall
+sudo service pf start       # start firewall
 ```
  
-Again, copy certificate files `privkey.pem` and `cert.pem` to `~/.local/share/certs`.
+Again, copy certificate files `privkey.pem` and `cert.pem` to
+`~/.local/share/certs`.
  
 ---
  
 ## Upgrading
  
-From time to time there can be updates to perguntations, python packages and javascript libraries.
-
-Python packages can be upgraded independently of the rest using pip:
+From time to time there can be updates to perguntations, python packages and
+javascript libraries.  Python packages can be upgraded independently of the
+rest using pip:
  
 ```sh
-pip list --outdated         # lists upgradable packages
+pip list --outdated --user  # lists upgradable packages
 pip install -U something    # upgrade something
 ```
  
+Attention: don't upgrade blindly. Some upgrades can be incompatible with
+perguntations and break things! All upgrades require testing of all the
+funcionalities.
+
 To upgrade perguntations and javascript libraries do:
  
 ```sh
 cd perguntations
 git pull                    # get latest version of perguntations
 npm update                  # get latest versions of javascript libraries
-pip3 install -U .           # upgrade perguntations
+pip install -U .            # upgrade perguntations
 ```
  
 ## Troubleshooting
  
 - The server tries to run `python3` so this command must be accessible from
-user accounts. Currently, the minimum supported python version is 3.7.
-
+  user accounts. Currently, the minimum supported python version is 3.8.
 - If you are getting any `UnicodeEncodeError` type of errors that's because the
-terminal is not supporting UTF-8.
-This error may occur when a unicode character is printed to the screen by the
-server or, when running question generator or correction scripts, a message is
-piped between the server and the scripts that includes unicode characters.
-Try running `locale` on the terminal and see if there are any error messages.
-Solutions:
+  terminal is not supporting UTF-8.  This error may occur when a unicode
+  character is printed to the screen by the server or, when running question
+  generator or correction scripts, a message is piped between the server and
+  the scripts that includes unicode characters.  Try running `locale` on the
+  terminal and see if there are any error messages.
+  Solutions:
+
   - debian: `sudo dpkg-reconfigure locales` and select your UTF-8 locales.
   - FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
  
@@ -14,9 +14,6 @@ database: students.db
 # Directory where the submitted and corrected test are stored for later review.
 answers_dir: ans
  
-# Server used to compile & execute code
-jobe_server: 192.168.1.85
-
 # --- optional settings: -----------------------------------------------------
  
 # Title of this test, e.g. course name, year or test number
@@ -41,43 +38,46 @@ autocorrect: true
 # (default: true)
 show_points: true
  
-# scale final grade to an interval, e.g. [0, 20], keeping the relative weight
-# of the points declared in the questions below.
+# Scale the points of the questions so that the final grade is in the given
+# interval.
 # (default: no scaling, just use question points)
-scale: [0, 5]
+scale: [0, 20]
  
  
 # ----------------------------------------------------------------------------
-# Base path applied to the questions files and all the scripts
-# including question generators and correctors.
-# Either absolute path or relative to current directory can be used.
-questions_dir: .
-
-# (optional) List of files containing questions in yaml format.
-# Selected questions will be obtained from these files.
-# If undefined, all yaml files in questions_dir are loaded (not recommended).
+# Files to import. Each file contains a list of questions in yaml format.
 files:
   - questions/questions-tutorial.yaml
  
 # This is the list of questions that will make up the test.
 # The order is preserved.
-# There are several ways to define each question (explained below).
+# Each question is a dictionary with a question `ref` or a list of `ref`.
+# If a list is given, one question will be choosen randomly to each student.
+# The `points` for each question is optional and is 1.0 by default for normal
+# questions. Informative type of "questions" will have 0.0 points.
+# Points are automatically scaled if `scale` key is defined.
 questions:
   - ref: tut-test
-  - tut-questions
+  - ref: tut-questions
+
+  # these will have 1.0 points
+  - ref: tut-radio
+  - ref: tut-checkbox
+  - ref: tut-text
+  - ref: tut-text-regex
+  - ref: tut-numeric-interval
  
-  - tut-radio
-  - tut-checkbox
-  - tut-text
-  - tut-text-regex
-  - tut-numeric-interval
+  # this question will have 2.0 points
   - ref: tut-textarea
     points: 2.0
  
-  - tut-information
-  - tut-success
-  - tut-warning
-  - [tut-alert1, tut-alert2]
-  - tut-generator
-  - tut-yamllint
-  # - tut-code
+  # these will have 0.0 points:
+  - ref: tut-information
+  - ref: tut-success
+  - ref: tut-warning
+
+  # choose one from the list:
+  - ref: [tut-alert1, tut-alert2]
+
+  - ref: tut-generator
+  - ref: tut-yamllint
@@ -0,0 +1,77 @@
+#!/usr/bin/env python3
+
+'''
+Example of a question generator.
+Arguments are read from stdin.
+'''
+
+from random import randint
+import sys
+
+# read two arguments from the field `args` specified in the question yaml file
+a, b = (int(n) for n in sys.argv[1:])
+
+x = randint(a, b)
+y = randint(a, b)
+r = x + y
+
+print(f"""---
+type: text
+title: Geradores de perguntas
+text: |
+
+  As perguntas podem ser estáticas (como as que vimos até aqui), ou serem
+  geradas dinâmicamente por um programa externo. Para gerar uma pergunta, o
+  programa deve escrever texto no `stdout` em formato `yaml` tal como os
+  exemplos das perguntas estáticas dos tipos apresentados anteriormente. Pode
+  também receber argumentos de linha de comando para parametrizar a pergunta.
+  Aqui está um exemplo de uma pergunta gerada por um script python:
+
+  ```python
+      #!/usr/bin/env python3
+
+      from random import randint
+      import sys
+
+      a, b = (int(n) for n in sys.argv[1:])   # argumentos da linha de comando
+
+      x = randint(a, b)   # número inteiro no intervalo a..b
+      y = randint(a, b)   # número inteiro no intervalo a..b
+      r = x + y           # calcula resultado correcto
+
+      print(f'''---
+      type: text
+      title: Contas de somar
+      text: |
+        Calcule o resultado de ${{x}} + {{y}}$.
+      correct: '{{r}}'
+      solution: |
+        A solução é {{r}}.''')
+  ```
+
+  Este script deve ter permissões para poder ser executado no terminal.
+  Podemos testar o programa no terminal `./gen-somar.py 1 100` e verificar que
+  o output é uma pergunta válida em formato `yaml`. Agora é necessário indicar
+  que este script deve ser usado para gerar uma pergunta.
+
+  Uma pergunta gerada por um programa externo é declarada com
+
+  ```yaml
+      - type: generator
+        ref: gen-somar
+        script: gen-somar.py
+        # argumentos opcionais
+        args: [1, 100]
+  ```
+
+  O programa pode receber uma lista de argumentos de linha de comando
+  declarados em `args`.
+
+  ---
+
+  Calcule o resultado de ${x} + {y}$.
+
+  Os números foram gerados aleatoriamente no intervalo de {a} a {b}.
+correct: '{r}'
+solution: |
+  A solução é {r}.""")
@@ -1,77 +0,0 @@
-#!/usr/bin/env python3
-
-'''
-Example of a question generator.
-Arguments are read from stdin.
-'''
-
-from random import randint
-import sys
-
-# read two arguments from the field `args` specified in the question yaml file
-a, b = (int(n) for n in sys.argv[1:])
-
-x = randint(a, b)
-y = randint(a, b)
-r = x + y
-
-print(f"""---
-type: text
-title: Geradores de perguntas
-text: |
-
-  As perguntas podem ser estáticas (como as que vimos até aqui), ou serem
-  geradas dinâmicamente por um programa externo. Para gerar uma pergunta, o
-  programa deve escrever texto no `stdout` em formato `yaml` tal como os
-  exemplos das perguntas estáticas dos tipos apresentados anteriormente. Pode
-  também receber argumentos de linha de comando para parametrizar a pergunta.
-  Aqui está um exemplo de uma pergunta gerada por um script python:
-
-  ```python
-      #!/usr/bin/env python3
-
-      from random import randint
-      import sys
-
-      a, b = (int(n) for n in sys.argv[1:])   # argumentos da linha de comando
-
-      x = randint(a, b)   # número inteiro no intervalo a..b
-      y = randint(a, b)   # número inteiro no intervalo a..b
-      r = x + y           # calcula resultado correcto
-
-      print(f'''---
-      type: text
-      title: Contas de somar
-      text: |
-        Calcule o resultado de ${{x}} + {{y}}$.
-      correct: '{{r}}'
-      solution: |
-        A solução é {{r}}.''')
-  ```
-
-  Este script deve ter permissões para poder ser executado no terminal.
-  Podemos testar o programa no terminal `./gen-somar.py 1 100` e verificar que
-  o output é uma pergunta válida em formato `yaml`. Agora é necessário indicar
-  que este script deve ser usado para gerar uma pergunta.
-
-  Uma pergunta gerada por um programa externo é declarada com
-
-  ```yaml
-      - type: generator
-        ref: gen-somar
-        script: gen-somar.py
-        # argumentos opcionais
-        args: [1, 100]
-  ```
-
-  O programa pode receber uma lista de argumentos de linha de comando
-  declarados em `args`.
-
-  ---
-
-  Calcule o resultado de ${x} + {y}$.
-
-  Os números foram gerados aleatoriamente no intervalo de {a} a {b}.
-correct: '{r}'
-solution: |
-  A solução é {r}.""")
@@ -20,24 +20,20 @@
     database: students.db      # base de dados previamente criada com initdb
     answers_dir: ans           # directório onde ficam os testes dos alunos
  
-    # opcional
+    # opcionais
     duration: 60               # duração da prova em minutos (default: inf)
     autosubmit: true           # submissão automática (default: false)
     show_points: true          # mostra cotação das perguntas (default: true)
-    scale: [0, 20]             # limites inferior e superior da escala (default: [0,20])
-    scale_points: true         # normaliza cotações para a escala definida
-    jobe_server: moodle-jobe.uevora.pt  # server used to compile & execute code
-    debug: false               # mostra informação de debug no browser
+    scale: [0, 20]             # normaliza cotações para o intervalo indicado.
+                               # não normaliza por defeito (default: None)
  
     # --------------------------------------------------------------------------
-    questions_dir: ~/topics    # raíz da árvore de directórios das perguntas
-
     # Ficheiros de perguntas a importar (relativamente a `questions_dir`)
     files:
       - tabelas.yaml
-      - topic_A/questions.yaml
-      - topic_B/part_1/questions.yaml
-      - topic_B/part_2/questions.yaml
+      - topic1/questions.yaml
+      - topic2/part1/questions.yaml
+      - topic2/part2/questions.yaml
  
     # --------------------------------------------------------------------------
     # Especificação das perguntas do teste e respectivas cotações.
@@ -50,13 +46,10 @@
       - ref: pergunta2
         points: 2.0
  
-      # a cotação é 1.0 por defeito
+      # por defeinto, a cotação da pergunta é 1.0 valor
       - ref: pergunta3
  
-      # uma string (não dict), é interpretada como referência
-      - tabela-auxiliar
-
-      # escolhe aleatoriamente uma das variantes
+      # escolhe aleatoriamente uma das variantes da pergunta
       - ref: [pergunta3a, pergunta3b]
         points: 0.5
  
@@ -96,8 +89,7 @@
       text: |
         Quando o texto da pergunta tem várias linhas, dá jeito usar o símbolo
         `|` de pipe, para indicar que tudo o que estiver indentado faz parte do
-        texto.
-        É o caso desta pergunta.
+        texto. É o caso desta pergunta.
  
         O texto das perguntas é escrito em `markdown` e suporta fórmulas em
         LaTeX.
@@ -105,8 +97,8 @@
     #---------------------------------------------------------------------------
     ```
  
-    As chaves são usadas para construir o teste e não se podem repetir, mesmo em
-    ficheiros diferentes.
+    As chaves são usadas para construir o teste e não se podem repetir, mesmo
+    em ficheiros diferentes.
     De seguida mostram-se exemplos dos vários tipos de perguntas.
  
 # ----------------------------------------------------------------------------
@@ -431,6 +423,10 @@
     pode estar previamente preenchida como neste caso (use `answer: texto`).
   correct: correct/correct-question.py
   timeout: 5
+  tests_right:
+    - 'red green blue'
+  # tests_wrong:
+  #   - 'blue gray yellow'
  
 # ---------------------------------------------------------------------------
 - type: information
@@ -504,6 +500,7 @@
             return 0;   // comentario
         }
         ```
+
     ```
  
 # ---------------------------------------------------------------------------
@@ -586,7 +583,7 @@
 # ----------------------------------------------------------------------------
 - type: generator
   ref: tut-generator
-  script: generators/generate-question.py
+  script: generate-question.py
   args: [1, 100]
  
 # ----------------------------------------------------------------------------
 [mypy]
-python_version = 3.7
-# warn_return_any = True
-# warn_unused_configs = True
-
-[mypy-sqlalchemy.*]
+python_version = 3.9
 ignore_missing_imports = True
  
-[mypy-pygments.*]
-ignore_missing_imports = True
  
-[mypy-bcrypt.*]
-ignore_missing_imports = True
+; [mypy-setuptools.*]
+; ignore_missing_imports = True
  
-[mypy-mistune.*]
-ignore_missing_imports = True
+; [mypy-sqlalchemy.*]
+; ignore_missing_imports = True
+
+; [mypy-pygments.*]
+; ignore_missing_imports = True
+
+; [mypy-mistune.*]
+; ignore_missing_imports = True
@@ -2,12 +2,12 @@
     "description": "Javascript libraries required to run the server",
     "email": "mjsb@uevora.pt",
     "dependencies": {
-        "@fortawesome/fontawesome-free": "^5.15.1",
-        "bootstrap": "^4.5",
-        "codemirror": "^5.58.1",
+        "@fortawesome/fontawesome-free": "^5.15.3",
+        "bootstrap": "^4.6.0",
+        "codemirror": "^5.61.1",
         "datatables": "^1.10",
-        "jquery": "^3.5.1",
-        "mathjax": "^3.1.2",
-        "underscore": "^1.11.0"
+        "jquery": "^3.6.0",
+        "mathjax": "^3.1.4",
+        "underscore": "^1.13.1"
     }
 }
-# Copyright (C) 2020 Miguel Barão
+# Copyright (C) 2022 Miguel Barão
 #
 # THE MIT License
 #
@@ -32,10 +32,10 @@ proof of submission and for review.
 '''
  
 APP_NAME = 'perguntations'
-APP_VERSION = '2021.02.dev1'
-APP_DESCRIPTION = __doc__
+APP_VERSION = '2022.01.dev1'
+APP_DESCRIPTION = str(__doc__)
  
 __author__ = 'Miguel Barão'
-__copyright__ = 'Copyright 2020, Miguel Barão'
+__copyright__ = 'Copyright 2022, Miguel Barão'
 __license__ = 'MIT license'
 __version__ = APP_VERSION
 '''
-Main application module
+File:           perguntations/app.py
+Description:    Main application logic.
 '''
  
  
 # python standard libraries
 import asyncio
-from contextlib import contextmanager  # `with` statement in db sessions
 import csv
 import io
 import json
 import logging
-from os import path
+import os
+from typing import Optional
  
 # installed packages
 import bcrypt
-from sqlalchemy import create_engine, exc
-from sqlalchemy.orm import sessionmaker
+from sqlalchemy import create_engine, select
+from sqlalchemy.exc import OperationalError, NoResultFound, IntegrityError
+from sqlalchemy.orm import Session
+import yaml
  
 # this project
-from perguntations.models import Student, Test, Question
-from perguntations.tools import load_yaml
-from perguntations.testfactory import TestFactory, TestFactoryException
-import perguntations.test
-from perguntations.questions import question_from
+from .models import Student, Test, Question
+from .tools import load_yaml
+from .testfactory import TestFactory, TestFactoryException
+from .test import Test as TestInstance
+from .questions import question_from
  
+# setup logger for this module
 logger = logging.getLogger(__name__)
  
-
-# ============================================================================
-class AppException(Exception):
-    '''Exception raised in this module'''
-
-
-# ============================================================================
-# helper functions
-# ============================================================================
-async def check_password(try_pw, hashed_pw):
+async def check_password(password: str, hashed: bytes) -> bool:
     '''check password in executor'''
-    try_pw = try_pw.encode('utf-8')
     loop = asyncio.get_running_loop()
-    hashed = await loop.run_in_executor(None, bcrypt.hashpw, try_pw, hashed_pw)
-    return hashed_pw == hashed
-
+    return await loop.run_in_executor(None, bcrypt.checkpw,
+            password.encode('utf-8'), hashed)
  
-async def hash_password(password):
-    '''hash password in executor'''
+async def hash_password(password: str) -> bytes:
+    '''get hash for password'''
     loop = asyncio.get_running_loop()
     return await loop.run_in_executor(None, bcrypt.hashpw,
-                                      password.encode('utf-8'),
-                                      bcrypt.gensalt())
+            password.encode('utf-8'), bcrypt.gensalt())
  
+# ============================================================================
+class AppException(Exception):
+    '''Exception raised in this module'''
  
 # ============================================================================
 # main application
 # ============================================================================
 class App():
     '''
-    This is the main application
-    state:
-      self.Session
-      self.online - {uid:
-                         {'student':{...}, 'test': {...}},
-                          ...
-                     }
-      self.allowd - {'123', '124', ...}
-      self.testfactory - TestFactory
+    Main application
     '''
  
     # ------------------------------------------------------------------------
-    @contextmanager
-    def _db_session(self):
-        '''
-        helper to manage db sessions using the `with` statement, for example:
-        with self._db_session() as s:  s.query(...)
-        '''
-        session = self.Session()
-        try:
-            yield session
-            session.commit()
-        except exc.SQLAlchemyError:
-            logger.error('DB rollback!!!')
-            session.rollback()
-            raise
-        finally:
-            session.close()
+    def __init__(self, config):
+        self.debug = config['debug']
+        self._make_test_factory(config['testfile'])
+        self._db_setup()  # setup engine and load all students
  
-    # ------------------------------------------------------------------------
-    def __init__(self, conf) -> None:
-        self.online = dict()    # {uid: {'student':{...}, 'test': {...}}, ...}
-        self.allowed = set()    # '0' is hardcoded to allowed elsewhere
-        self.unfocus = set()    # set of students that have no browser focus
-        self.area = dict()      # {uid: percent_area}
-        self.pregenerated_tests = []  # list of tests to give to students
-
-        self._make_test_factory(conf)
-
-        # connect to database and check registered students
-        dbfile = self.testfactory['database']
-        database = f'sqlite:///{path.expanduser(dbfile)}'
-        engine = create_engine(database, echo=False)
-        self.Session = sessionmaker(bind=engine)
-        try:
-            with self._db_session() as sess:
-                num = sess.query(Student).filter(Student.id != '0').count()
-        except Exception as exc:
-            raise AppException(f'Database unusable {dbfile}.') from exc
-        logger.info('Database "%s" has %s students.', dbfile, num)
-
-        # command line option --allow-all
-        if conf['allow_all']:
-            self.allow_all_students()
-        elif conf['allow_list'] is not None:
-            self.allow_list(conf['allow_list'])
-        else:
-            logger.info('Students not yet allowed to login.')
+        # FIXME get_event_loop will be deprecated in python3.10
+        asyncio.get_event_loop().run_until_complete(self._assign_tests())
  
-        # pre-generate tests for allowed students
-        if self.allowed:
-            logger.info('Generating %d tests. May take awhile...',
-                        len(self.allowed))
-            self._pregenerate_tests(len(self.allowed))
+        # command line options:  --allow-all, --allow-list filename
+        if config['allow_all']:
+            self.allow_all_students()
+        elif config['allow_list'] is not None:
+            self.allow_from_list(config['allow_list'])
         else:
-            logger.info('No tests were generated.')
+            logger.info('Students not allowed to login')
  
-        if conf['correct']:
+        if config['correct']:
             self._correct_tests()
  
     # ------------------------------------------------------------------------
-    def _correct_tests(self) -> None:
-        with self._db_session() as sess:
-            # Find which tests have to be corrected.
-            # already corrected tests are not included.
-            dbtests = sess.query(Test)\
-                    .filter(Test.ref == self.testfactory['ref'])\
-                    .filter(Test.state == "SUBMITTED")\
-                    .all()
-
-            logger.info('Correcting %d tests...', len(dbtests))
-            for dbtest in dbtests:
-                try:
-                    with open(dbtest.filename) as file:
-                        testdict = json.load(file)
-                except FileNotFoundError:
-                    logger.error('File not found: %s', dbtest.filename)
-                    continue
+    def _db_setup(self) -> None:
+        '''
+        Create database engine and checks for admin and students
+        '''
+        dbfile = os.path.expanduser(self._testfactory['database'])
+        logger.debug('Checking database "%s"...', dbfile)
+        if not os.path.exists(dbfile):
+            raise AppException('No database, use "initdb" to create')
  
-                # creates a class Test with the methods to correct it
-                # the questions are still dictionaries, so we have to call
-                # question_from() to produce Question() instances that can be
-                # corrected. Finally the test can be corrected.
-                test = perguntations.test.Test(testdict)
-                test['questions'] = [question_from(q) for q in test['questions']]
-                test.correct()
-                logger.info('Student %s:  grade = %f', test['student']['number'], test['grade'])
+        # connect to database and check for admin & registered students
+        self._engine = create_engine(f'sqlite:///{dbfile}', future=True)
+        try:
+            with Session(self._engine, future=True) as session:
+                query = select(Student.id, Student.name)\
+                        .where(Student.id != '0')
+                dbstudents = session.execute(query).all()
+                session.execute(select(Student).where(Student.id == '0')).one()
+        except NoResultFound:
+            msg = 'Database has no administrator (user "0")'
+            logger.error(msg)
+            raise AppException(msg) from None
+        except OperationalError:
+            msg = f'Database "{dbfile}" unusable'
+            logger.error(msg)
+            raise AppException(msg) from None
+        logger.info('Database has %d students', len(dbstudents))
  
-                # save JSON file (overwriting the old one)
-                uid = test['student']['number']
-                ref = test['ref']
-                finish_time = test['finish_time']
-                answers_dir = test['answers_dir']
-                fname = f'{uid}--{ref}--{finish_time}.json'
-                fpath = path.join(answers_dir, fname)
-                test.save_json(fpath)
-                logger.info('%s saved JSON file.', uid)
+        self._students = {uid: {
+                'name': name,
+                'state': 'offline',
+                'test': None,
+            } for uid, name in dbstudents}
  
-                # update database
-                dbtest.grade = test['grade']
-                dbtest.state = test['state']
-                dbtest.questions = [
-                    Question(
-                        number=n,
-                        ref=q['ref'],
-                        grade=q['grade'],
-                        comment=q.get('comment', ''),
-                        starttime=str(test['start_time']),
-                        finishtime=str(test['finish_time']),
-                        test_id=test['ref']
-                        )
-                    for n, q in enumerate(test['questions'])
-                    ]
-                logger.info('%s database updated.', uid)
+    # ------------------------------------------------------------------------
+    async def _assign_tests(self) -> None:
+        '''Generate tests for all students that don't yet have a test'''
+        logger.info('Generating tests...')
+        for student in self._students.values():
+            if student.get('test', None) is None:
+                student['test'] = await self._testfactory.generate()
+        logger.info('Tests assigned to all students')
  
     # ------------------------------------------------------------------------
-    async def login(self, uid, try_pw, headers=None):
-        '''login authentication'''
-        if uid not in self.allowed and uid != '0':      # not allowed
-            logger.warning('"%s" unauthorized.', uid)
-            return 'unauthorized'
-
-        with self._db_session() as sess:
-            name, hashed_pw = sess.query(Student.name, Student.password)\
-                                 .filter_by(id=uid)\
-                                 .one()
-
-        if hashed_pw == '':              # update password on first login
-            await self.update_student_password(uid, try_pw)
-            pw_ok = True
-        else:                           # check password
-            pw_ok = await check_password(try_pw, hashed_pw)  # async bcrypt
-
-        if not pw_ok:        # wrong password
-            logger.info('"%s" wrong password.', uid)
+    async def login(self, uid: str, password: str, headers: dict) -> Optional[str]:
+        '''
+        Login authentication
+        If successful returns None, else returns an error message
+        '''
+        try:
+            with Session(self._engine, future=True) as session:
+                query = select(Student.password).where(Student.id == uid)
+                hashed = session.execute(query).scalar_one()
+        except NoResultFound:
+            logger.warning('"%s" does not exist', uid)
+            return 'nonexistent'
+
+        if uid != '0' and self._students[uid]['state'] != 'allowed':
+            logger.warning('"%s" login not allowed', uid)
+            return 'not_allowed'
+
+        if hashed == '':  # set password on first login
+            await self.set_password(uid, password)
+        elif not await check_password(password, hashed):
+            logger.info('"%s" wrong password', uid)
             return 'wrong_password'
  
         # success
-        self.allowed.discard(uid)  # remove from set of allowed students
-
-        if uid in self.online:
-            logger.warning('"%s" login again from %s (reusing state).',
-                           uid, headers['remote_ip'])
-            # FIXME invalidate previous login
+        if uid == '0':
+            logger.info('Admin login from %s', headers['remote_ip'])
         else:
-            self.online[uid] = {'student': {
-                                    'name': name,
-                                    'number': uid,
-                                    'headers': headers}}
-            logger.info('"%s" login from %s.', uid, headers['remote_ip'])
+            student = self._students[uid]
+            student['test'].start(uid)
+            student['state'] = 'online'
+            student['headers'] = headers
+            student['unfocus'] = False
+            student['area'] = 0.0
+            logger.info('"%s" login from %s', uid, headers['remote_ip'])
+        return None
  
     # ------------------------------------------------------------------------
-    def logout(self, uid):
+    async def set_password(self, uid: str, password: str) -> None:
+        '''change password on the database'''
+        with Session(self._engine, future=True) as session:
+            query = select(Student).where(Student.id == uid)
+            student = session.execute(query).scalar_one()
+            student.password = await hash_password(password) if password else ''
+            session.commit()
+        logger.info('"%s" password updated', uid)
+
+    # ------------------------------------------------------------------------
+    def logout(self, uid: str) -> None:
         '''student logout'''
-        self.online.pop(uid, None)  # remove from dict if exists
-        logger.info('"%s" logged out.', uid)
+        student = self._students.get(uid, None)
+        if student is not None:
+            # student['test'] = None
+            student['state'] = 'offline'
+            student.pop('headers', None)
+            student.pop('unfocus', None)
+            student.pop('area', None)
+            logger.info('"%s" logged out', uid)
  
     # ------------------------------------------------------------------------
-    def _make_test_factory(self, conf):
+    def _make_test_factory(self, filename: str) -> None:
         '''
         Setup a factory for the test
         '''
  
         # load configuration from yaml file
-        logger.info('Loading test configuration "%s".', conf["testfile"])
         try:
-            testconf = load_yaml(conf['testfile'])
-        except Exception as exc:
-            msg = 'Error loading test configuration YAML.'
-            logger.critical(msg)
+            testconf = load_yaml(filename)
+            testconf['testfile'] = filename
+        except (OSError, yaml.YAMLError) as exc:
+            msg = f'Cannot read test configuration "{filename}"'
+            logger.error(msg)
             raise AppException(msg) from exc
  
-        # command line options override configuration
-        testconf.update(conf)
-
-        # start test factory
+        # make test factory
         logger.info('Running test factory...')
         try:
-            self.testfactory = TestFactory(testconf)
+            self._testfactory = TestFactory(testconf)
         except TestFactoryException as exc:
-            logger.critical(exc)
+            logger.error(exc)
             raise AppException('Failed to create test factory!') from exc
  
     # ------------------------------------------------------------------------
-    def _pregenerate_tests(self, num):  # TODO needs improvement
-        event_loop = asyncio.get_event_loop()
-        self.pregenerated_tests += [
-                event_loop.run_until_complete(self.testfactory.generate())
-                for _ in range(num)]
-
-    # ------------------------------------------------------------------------
-    async def get_test_or_generate(self, uid):
-        '''get current test or generate a new one'''
-        try:
-            student = self.online[uid]
-        except KeyError as exc:
-            msg = f'"{uid}" is not online. get_test_or_generate() FAILED'
-            logger.error(msg)
-            raise AppException(msg) from exc
-
-        # get current test. if test does not exist then generate a new one
-        if not 'test' in student:
-            await self._new_test(uid)
-
-        return student['test']
-
-    def get_test(self, uid):
-        '''get test from online student or raise exception'''
-        return self.online[uid]['test']
-
-    # ------------------------------------------------------------------------
-    async def _new_test(self, uid):
-        '''
-        assign a test to a given student. if there are pregenerated tests then
-        use one of them, otherwise generate one.
-        the student must be online
-        '''
-        student = self.online[uid]['student']  # {'name': ?, 'number': ?}
-
-        try:
-            test = self.pregenerated_tests.pop()
-        except IndexError:
-            logger.info('"%s" generating new test...', uid)
-            test = await self.testfactory.generate()
-            logger.info('"%s" test is ready.', uid)
-        else:
-            logger.info('"%s" using a pregenerated test.', uid)
-
-        test.start(student)           # student signs the test
-        self.online[uid]['test'] = test
-
-    # ------------------------------------------------------------------------
-    async def submit_test(self, uid, ans):
+    async def submit_test(self, uid, ans) -> None:
         '''
         Handles test submission and correction.
  
         ans is a dictionary {question_index: answer, ...} with the answers for
         the complete test. For example:  {0:'hello', 1:[1,2]}
         '''
-        test = self.online[uid]['test']
+        if self._students[uid]['state'] != 'online':
+            logger.warning('"%s" INVALID SUBMISSION! STUDENT NOT ONLINE', uid)
+            return
  
         # --- submit answers and correct test
+        logger.info('"%s" submitted %d answers', uid, len(ans))
+        test = self._students[uid]['test']
         test.submit(ans)
-        logger.info('"%s" submitted %d answers.', uid, len(ans))
  
         if test['autocorrect']:
             await test.correct_async()
-            logger.info('"%s" grade = %g points.', uid, test['grade'])
+            logger.info('"%s" grade = %g points', uid, test['grade'])
  
         # --- save test in JSON format
         fname = f'{uid}--{test["ref"]}--{test["finish_time"]}.json'
-        fpath = path.join(test['answers_dir'], fname)
+        fpath = os.path.join(test['answers_dir'], fname)
         test.save_json(fpath)
-        logger.info('"%s" saved JSON.', uid)
+        logger.info('"%s" saved JSON', uid)
  
         # --- insert test and questions into the database
         #     only corrected questions are added
@@ -348,13 +253,62 @@ class App():
                 for n, q in enumerate(test['questions'])
                 ]
  
-        with self._db_session() as sess:
-            sess.add(test_row)
-        logger.info('"%s" database updated.', uid)
+        with Session(self._engine, future=True) as session:
+            session.add(test_row)
+            session.commit()
+        logger.info('"%s" database updated', uid)
  
     # ------------------------------------------------------------------------
-    def get_student_grade(self, uid):
-        return self.online[uid]['test'].get('grade', None)
+    def _correct_tests(self) -> None:
+        with Session(self._engine, future=True) as session:
+            # Find which tests have to be corrected
+            query = select(Test) \
+                    .where(Test.ref == self._testfactory['ref']) \
+                    .where(Test.state == "SUBMITTED")
+            dbtests = session.execute(query).scalars().all()
+            if not dbtests:
+                logger.info('No tests to correct')
+                return
+
+            logger.info('Correcting %d tests...', len(dbtests))
+            for dbtest in dbtests:
+                try:
+                    with open(dbtest.filename) as file:
+                        testdict = json.load(file)
+                except OSError:
+                    logger.error('Failed: %s', dbtest.filename)
+                    continue
+
+                # creates a class Test with the methods to correct it
+                # the questions are still dictionaries, so we have to call
+                # question_from() to produce Question() instances that can be
+                # corrected. Finally the test can be corrected.
+                test = TestInstance(testdict)
+                test['questions'] = [question_from(q) for q in test['questions']]
+                test.correct()
+                logger.info('  %s:  %f', test['student'], test['grade'])
+
+                # save JSON file (overwriting the old one)
+                uid = test['student']
+                test.save_json(dbtest.filename)
+                logger.debug('%s saved JSON file', uid)
+
+                # update database
+                dbtest.grade = test['grade']
+                dbtest.state = test['state']
+                dbtest.questions = [
+                    Question(
+                        number=n,
+                        ref=q['ref'],
+                        grade=q['grade'],
+                        comment=q.get('comment', ''),
+                        starttime=str(test['start_time']),
+                        finishtime=str(test['finish_time']),
+                        test_id=test['ref']
+                        ) for n, q in enumerate(test['questions'])
+                    ]
+            session.commit()
+            logger.info('Database updated')
  
     # ------------------------------------------------------------------------
     # def giveup_test(self, uid):
@@ -366,7 +320,7 @@ class App():
     #     fields = (test['student']['number'], test['ref'],
     #               str(test['finish_time']))
     #     fname = '--'.join(fields) + '.json'
-    #     fpath = path.join(test['answers_dir'], fname)
+    #     fpath = os.path.join(test['answers_dir'], fname)
     #     test.save_json(fpath)
  
     #     # insert test into database
@@ -381,11 +335,11 @@ class App():
     #                       state=test['state'],
     #                       comment=''))
  
-    #     logger.info('"%s" gave up.', uid)
+    #     logger.info('"%s" gave up', uid)
     #     return test
  
     # ------------------------------------------------------------------------
-    def event_test(self, uid, cmd, value):
+    def register_event(self, uid, cmd, value):
         '''handles browser events the occur during the test'''
         if cmd == 'focus':
             if value:
@@ -395,202 +349,200 @@ class App():
         elif cmd == 'size':
             self._set_screen_area(uid, value)
  
-    # ------------------------------------------------------------------------
-    # --- GETTERS
-    # ------------------------------------------------------------------------
+    # ========================================================================
+    #   GETTERS
+    # ========================================================================
+    def get_test(self, uid: str) -> Optional[dict]:
+        '''return student test'''
+        return self._students[uid]['test']
  
-    # def get_student_name(self, uid):
-    #     return self.online[uid]['student']['name']
+    # ------------------------------------------------------------------------
+    def get_name(self, uid: str) -> str:
+        '''return name of student'''
+        return self._students[uid]['name']
  
-    def get_questions_csv(self):
-        '''generates a CSV with the grades of the test'''
-        test_ref = self.testfactory['ref']
-        with self._db_session() as sess:
-            questions = sess.query(Test.id, Test.student_id, Test.starttime,
-                                   Question.number, Question.grade)\
-                            .join(Question)\
-                            .filter(Test.ref == test_ref)\
-                            .all()
-
-            qnums = set()  # keeps track of all the questions in the test
-            tests = {}      # {test_id: {student_id, starttime, 0: grade, ...}}
-            for question in questions:
-                test_id, student_id, starttime, num, grade = question
-                default_test_id = {'Aluno': student_id, 'Início': starttime}
-                tests.setdefault(test_id, default_test_id)[num] = grade
-                qnums.add(num)
+    # ------------------------------------------------------------------------
+    def get_test_config(self) -> dict:
+        '''return brief test configuration to use as header in /admin'''
+        return {'title': self._testfactory['title'],
+                'ref': self._testfactory['ref'],
+                'filename': self._testfactory['testfile'],
+                'database': self._testfactory['database'],
+                'answers_dir': self._testfactory['answers_dir']
+                }
  
+    # ------------------------------------------------------------------------
+    def get_test_csv(self):
+        '''generates a CSV with the grades of the test currently running'''
+        test_ref = self._testfactory['ref']
+        with Session(self._engine, future=True) as session:
+            query = select(Test.student_id, Test.grade,
+                           Test.starttime, Test.finishtime)\
+                    .where(Test.ref == test_ref)\
+                    .order_by(Test.student_id)
+            tests = session.execute(query).all()
         if not tests:
             logger.warning('Empty CSV: there are no tests!')
             return test_ref, ''
  
-        cols = ['Aluno', 'Início'] + list(qnums)
-
         csvstr = io.StringIO()
-        writer = csv.DictWriter(csvstr, fieldnames=cols, restval=None,
-                                delimiter=';', quoting=csv.QUOTE_ALL)
-        writer.writeheader()
-        writer.writerows(tests.values())
+        writer = csv.writer(csvstr, delimiter=';', quoting=csv.QUOTE_ALL)
+        writer.writerow(('Aluno', 'Nota', 'Início', 'Fim'))
+        writer.writerows(tests)
         return test_ref, csvstr.getvalue()
  
-    def get_test_csv(self):
-        '''generates a CSV with the grades of the test currently running'''
-        test_ref = self.testfactory['ref']
-        with self._db_session() as sess:
-            tests = sess.query(Test.student_id,
-                                Test.grade,
-                                Test.starttime, Test.finishtime)\
-                         .filter(Test.ref == test_ref)\
-                         .order_by(Test.student_id)\
-                         .all()
+    # ------------------------------------------------------------------------
+    def get_detailed_grades_csv(self):
+        '''generates a CSV with the grades of the test'''
+        test_ref = self._testfactory['ref']
+        with Session(self._engine, future=True) as session:
+            query = select(Test.id, Test.student_id, Test.starttime,
+                           Question.number, Question.grade)\
+                    .join(Question)\
+                    .where(Test.ref == test_ref)
+            questions = session.execute(query).all()
+
+        cols = ['Aluno', 'Início']
+        tests = {}      # {test_id: {student_id, starttime, 0: grade, ...}}
+        for test_id, student_id, starttime, num, grade in questions:
+            default_test_id = {'Aluno': student_id, 'Início': starttime}
+            tests.setdefault(test_id, default_test_id)[num] = grade
+            if num not in cols:
+                cols.append(num)
  
         if not tests:
             logger.warning('Empty CSV: there are no tests!')
             return test_ref, ''
  
         csvstr = io.StringIO()
-        writer = csv.writer(csvstr, delimiter=';', quoting=csv.QUOTE_ALL)
-        writer.writerow(('Aluno', 'Nota', 'Início', 'Fim'))
-        writer.writerows(tests)
-
+        writer = csv.DictWriter(csvstr, fieldnames=cols, restval=None,
+                                delimiter=';', quoting=csv.QUOTE_ALL)
+        writer.writeheader()
+        writer.writerows(tests.values())
         return test_ref, csvstr.getvalue()
  
     # ------------------------------------------------------------------------
-    def get_student_grades_from_all_tests(self, uid):
-        '''get grades of student from all tests'''
-        with self._db_session() as sess:
-            return sess.query(Test.title, Test.grade, Test.finishtime)\
-                       .filter_by(student_id=uid)\
-                       .order_by(Test.finishtime)
-
     def get_json_filename_of_test(self, test_id):
         '''get JSON filename from database given the test_id'''
-        with self._db_session() as sess:
-            return sess.query(Test.filename)\
-                       .filter_by(id=test_id)\
-                       .scalar()
+        with Session(self._engine, future=True) as session:
+            query = select(Test.filename).where(Test.id == test_id)
+            return session.execute(query).scalar()
  
-    def get_student_grades_from_test(self, uid, testid):
+    # ------------------------------------------------------------------------
+    def get_grades(self, uid, ref):
         '''get grades of student for a given testid'''
-        with self._db_session() as sess:
-            return sess.query(Test.grade, Test.finishtime, Test.id)\
-                       .filter_by(student_id=uid)\
-                       .filter_by(ref=testid)\
-                       .all()
-
-    def get_students_state(self):
-        '''get list of states of every student'''
-        return [{
-            'uid': uid,
-            'name': name,
-            'allowed': uid in self.allowed,
-            'online': uid in self.online,
-            'start_time': self.online.get(uid, {}).get('test', {})
-                          .get('start_time', ''),
-            'password_defined': pw != '',
-            'unfocus': uid in self.unfocus,
-            'area': self.area.get(uid, None),
-            'grades': self.get_student_grades_from_test(
-                uid, self.testfactory['ref'])
-            } for uid, name, pw in self._get_all_students()]
-
-    # --- private methods ----------------------------------------------------
-    def _get_all_students(self):
-        '''get all students from database'''
-        with self._db_session() as sess:
-            return sess.query(Student.id, Student.name, Student.password)\
-                       .filter(Student.id != '0')\
-                       .order_by(Student.id)
-
-    # def get_allowed_students(self):
-    #     # set of 'uid' allowed to login
-    #     return self.allowed
-
-    # def get_file(self, uid, ref, key):
-    #     # get filename of (uid, ref, name) if declared in the question
-    #     t = self.get_student_test(uid)
-    #     for q in t['questions']:
-    #         if q['ref'] == ref and key in q['files']:
-    #             return path.abspath(path.join(q['path'], q['files'][key]))
+        with Session(self._engine, future=True) as session:
+            query = select(Test.grade, Test.finishtime, Test.id)\
+                    .where(Test.student_id == uid)\
+                    .where(Test.ref == ref)
+            grades = session.execute(query).all()
+        return [tuple(grade) for grade in grades]
  
     # ------------------------------------------------------------------------
-    # --- SETTERS
-    # ------------------------------------------------------------------------
-
-    def allow_student(self, uid):
+    def get_students_state(self) -> list:
+        '''get list of states of every student to show in /admin page'''
+        return [{   'uid': uid,
+                    'name': student['name'],
+                    'allowed': student['state'] == 'allowed',
+                    'online': student['state'] == 'online',
+                    'start_time': student.get('test', {}).get('start_time', ''),
+                    'unfocus': student.get('unfocus', False),
+                    'area': student.get('area', 1.0),
+                    'grades': self.get_grades(uid, self._testfactory['ref']) }
+                for uid, student in self._students.items()]
+
+    # ========================================================================
+    #   SETTERS
+    # ========================================================================
+    def allow_student(self, uid: str) -> None:
         '''allow a single student to login'''
-        self.allowed.add(uid)
-        logger.info('"%s" allowed to login.', uid)
+        self._students[uid]['state'] = 'allowed'
+        logger.info('"%s" allowed to login', uid)
  
-    def deny_student(self, uid):
+    # ------------------------------------------------------------------------
+    def deny_student(self, uid: str) -> None:
         '''deny a single student to login'''
-        self.allowed.discard(uid)
+        student = self._students[uid]
+        if student['state'] == 'allowed':
+            student['state'] = 'offline'
         logger.info('"%s" denied to login', uid)
  
-    def allow_all_students(self):
+    # ------------------------------------------------------------------------
+    def allow_all_students(self) -> None:
         '''allow all students to login'''
-        all_students = self._get_all_students()
-        self.allowed.update(s[0] for s in all_students)
-        logger.info('Allowed all %d students.', len(self.allowed))
+        for student in self._students.values():
+            student['state'] = 'allowed'
+        logger.info('Allowed %d students', len(self._students))
  
-    def deny_all_students(self):
+    # ------------------------------------------------------------------------
+    def deny_all_students(self) -> None:
         '''deny all students to login'''
         logger.info('Denying all students...')
-        self.allowed.clear()
+        for student in self._students.values():
+            if student['state'] == 'allowed':
+                student['state'] = 'offline'
+
+    # ------------------------------------------------------------------------
+    async def insert_new_student(self, uid: str, name: str) -> None:
+        '''insert new student into the database'''
+        with Session(self._engine, future=True) as session:
+            try:
+                session.add(Student(id=uid, name=name, password=''))
+                session.commit()
+            except IntegrityError:
+                logger.warning('"%s" already exists!', uid)
+                session.rollback()
+                return
+        logger.info('New student added: %s %s', uid, name)
+        self._students[uid] = {
+                'name': name,
+                'state': 'offline',
+                'test': await self._testfactory.generate(),
+                }
  
-    def allow_list(self, filename):
-        '''allow students listed in file (one number per line)'''
+    # ------------------------------------------------------------------------
+    def allow_from_list(self, filename: str) -> None:
+        '''allow students listed in text file (one number per line)'''
+        # parse list of students to allow (one number per line)
         try:
-            with open(filename, 'r') as file:
-                allowed_in_file = {s.strip() for s in file} - {''}
-        except Exception as exc:
+            with open(filename, 'r', encoding='utf-8') as file:
+                allowed = {line.strip() for line in file}
+                allowed.discard('')
+        except OSError as exc:
             error_msg = f'Cannot read file {filename}'
             logger.critical(error_msg)
             raise AppException(error_msg) from exc
  
-        enrolled = set(s[0] for s in self._get_all_students())  # in database
-        self.allowed.update(allowed_in_file & enrolled)
-        logger.info('Allowed %d students provided in "%s"', len(self.allowed),
-                    filename)
+        # update allowed state (missing are students allowed that don't exist)
+        missing = 0
+        for uid in allowed:
+            try:
+                self.allow_student(uid)
+            except KeyError:
+                logger.warning('Allowed student "%s" does not exist!', uid)
+                missing += 1
  
-        not_enrolled = allowed_in_file - enrolled
-        if not_enrolled:
-            logger.warning('  but found students not in the database:  %s',
-                           ', '.join(not_enrolled))
+        logger.info('Allowed %d students', len(allowed)-missing)
+        if missing:
+            logger.warning('  %d missing!', missing)
  
+    # ------------------------------------------------------------------------
     def _focus_student(self, uid):
         '''set student in focus state'''
-        self.unfocus.discard(uid)
+        self._students[uid]['unfocus'] = False
         logger.info('"%s" focus', uid)
  
+    # ------------------------------------------------------------------------
     def _unfocus_student(self, uid):
         '''set student in unfocus state'''
-        self.unfocus.add(uid)
+        self._students[uid]['unfocus'] = True
         logger.info('"%s" unfocus', uid)
  
+    # ------------------------------------------------------------------------
     def _set_screen_area(self, uid, sizes):
         '''set current browser area as detected in resize event'''
         scr_y, scr_x, win_y, win_x = sizes
         area = win_x * win_y / (scr_x * scr_y) * 100
-        self.area[uid] = area
+        self._students[uid]['area'] = area
         logger.info('"%s" area=%g%%, window=%dx%d, screen=%dx%d',
                     uid, area, win_x, win_y, scr_x, scr_y)
-
-    async def update_student_password(self, uid, password=''):
-        '''change password on the database'''
-        if password:
-            password = await hash_password(password)
-        with self._db_session() as sess:
-            student = sess.query(Student).filter_by(id=uid).one()
-            student.password = password
-        logger.info('"%s" password updated.', uid)
-
-    def insert_new_student(self, uid, name):
-        '''insert new student into the database'''
-        try:
-            with self._db_session() as sess:
-                sess.add(Student(id=uid, name=name, password=''))
-        except exc.SQLAlchemyError:
-            logger.error('Insert failed: student %s already exists?', uid)
-        else:
-            logger.info('New student: "%s", "%s"', uid, name)
@@ -13,17 +13,18 @@ from concurrent.futures import ThreadPoolExecutor
  
 # installed packages
 import bcrypt
-import sqlalchemy as sa
-import sqlalchemy.orm
+from sqlalchemy import create_engine, select
+from sqlalchemy.orm import Session
 from sqlalchemy.exc import IntegrityError
  
 # this project
-from perguntations.models import Base, Student
+from .models import Base, Student
  
  
 # ============================================================================
 def parse_commandline_arguments():
     '''Parse command line options'''
+
     parser = argparse.ArgumentParser(
         formatter_class=argparse.ArgumentDefaultsHelpFormatter,
         description='Insert new users into a database. Users can be imported '
@@ -104,7 +105,7 @@ def get_students_from_csv(filename):
  
  
 # ============================================================================
-def hashpw(student, password=None):
+def hashpw(student, password=None) -> None:
     '''replace password by hash for a single student'''
     print('.', end='', flush=True)
     if password is None:
@@ -115,12 +116,13 @@ def hashpw(student, password=None):
  
  
 # ============================================================================
-def insert_students_into_db(session, students):
+def insert_students_into_db(session, students) -> None:
     '''insert list of students into the database'''
     try:
         session.add_all([Student(id=s['uid'], name=s['name'], password=s['pw'])
                          for s in students])
         session.commit()
+
     except IntegrityError:
         print('!!! Integrity error. Users already in database. Aborted !!!\n')
         session.rollback()
@@ -129,11 +131,12 @@ def insert_students_into_db(session, students):
 # ============================================================================
 def show_students_in_database(session, verbose=False):
     '''get students from database'''
-    users = session.query(Student).all()
+    users = session.execute(select(Student)).scalars().all()
+    # users = session.query(Student).all()
+    total = len(users)
  
-    total_users = len(users)
     print('Registered users:')
-    if total_users == 0:
+    if total == 0:
         print('  -- none --')
     else:
         users.sort(key=lambda u: f'{u.id:>12}')  # sort by number
@@ -142,13 +145,13 @@ def show_students_in_database(session, verbose=False):
                 print(f'{user.id:>12}   {user.name}')
         else:
             print(f'{users[0].id:>12}   {users[0].name}')
-            if total_users > 1:
+            if total > 1:
                 print(f'{users[1].id:>12}   {users[1].name}')
-            if total_users > 3:
+            if total > 3:
                 print('           |   |')
-            if total_users > 2:
+            if total > 2:
                 print(f'{users[-1].id:>12}   {users[-1].name}')
-    print(f'Total: {total_users}.')
+    print(f'Total: {total}.')
  
  
 # ============================================================================
@@ -158,39 +161,45 @@ def main():
     args = parse_commandline_arguments()
  
     # --- database
-    print(f'Using database: {args.db}')
-    engine = sa.create_engine(f'sqlite:///{args.db}', echo=False)
+    print(f'Database: {args.db}')
+    engine = create_engine(f'sqlite:///{args.db}', echo=False, future=True)
     Base.metadata.create_all(engine)  # Criates schema if needed
-    SessionMaker = sqlalchemy.orm.sessionmaker(bind=engine)
-    session = SessionMaker()
+    session = Session(engine, future=True)
  
     # --- make list of students to insert
-    new_students = []
+    students = []
  
     if args.admin:
         print('Adding user: 0, Admin.')
-        new_students.append({'uid': '0', 'name': 'Admin'})
+        students.append({'uid': '0', 'name': 'Admin'})
  
     for csvfile in args.csvfile:
         print('Adding users from:', csvfile)
-        new_students.extend(get_students_from_csv(csvfile))
+        students.extend(get_students_from_csv(csvfile))
+
+    for csvfile in args.csvfile:
+        print('Adding users from:', csvfile)
+        students.extend(get_students_from_csv(csvfile))
  
     if args.add:
         for uid, name in args.add:
             print(f'Adding user: {uid}, {name}.')
-            new_students.append({'uid': uid, 'name': name})
+            students.append({'uid': uid, 'name': name})
  
     # --- insert new students
-    if new_students:
+    if students:
         print('Generating password hashes', end='')
-        with ThreadPoolExecutor() as executor:  # hashing in parallel
-            executor.map(lambda s: hashpw(s, args.pw), new_students)
-        print(f'\nInserting {len(new_students)}')
-        insert_students_into_db(session, new_students)
+        with ThreadPoolExecutor() as executor:  # hashing
+            executor.map(lambda s: hashpw(s, args.pw), students)
+        print(f'\nInserting {len(students)}')
+        insert_students_into_db(session, students)
  
     # --- update all students
     if args.update_all:
-        all_students = session.query(Student).filter(Student.id != '0').all()
+        all_students = session.execute(
+            select(Student).where(Student.id != '0')
+        ).scalars().all()
+
         print(f'Updating password of {len(all_students)} users', end='')
         for student in all_students:
             password = (args.pw or student.id).encode('utf-8')
@@ -203,9 +212,12 @@ def main():
     else:
         for student_id in args.update:
             print(f'Updating password of {student_id}')
-            student = session.query(Student).get(student_id)
-            password = (args.pw or student_id).encode('utf-8')
-            student.password = bcrypt.hashpw(password, bcrypt.gensalt())
+            student = session.execute(
+                select(Student).
+                where(Student.id == student_id)
+            ).scalar_one()
+            new_password = (args.pw or student_id).encode('utf-8')
+            student.password = bcrypt.hashpw(new_password, bcrypt.gensalt())
         session.commit()
  
     show_students_in_database(session, args.verbose)
 #!/usr/bin/env python3
  
 '''
-Main file that starts the application and the web server
+Start application and web server
 '''
  
  
@@ -10,20 +10,17 @@ import argparse
 import logging
 import logging.config
 import os
-from os import environ, path
 import ssl
 import sys
-# from typing import Any, Dict
  
 # this project
-from perguntations.app import App, AppException
-from perguntations.serve import run_webserver
-from perguntations.tools import load_yaml
-from perguntations import APP_NAME, APP_VERSION
-
+from .app import App, AppException
+from .serve import run_webserver
+from .tools import load_yaml
+from . import APP_NAME, APP_VERSION
  
 # ----------------------------------------------------------------------------
-def parse_cmdline_arguments():
+def parse_cmdline_arguments() -> argparse.Namespace:
     '''
     Get command line arguments
     '''
@@ -33,7 +30,6 @@ def parse_cmdline_arguments():
         'included with this software before running the server.')
     parser.add_argument('testfile',
                         type=str,
-                        # nargs='+',  # TODO
                         help='tests in YAML format')
     parser.add_argument('--allow-all',
                         action='store_true',
@@ -44,9 +40,6 @@ def parse_cmdline_arguments():
     parser.add_argument('--debug',
                         action='store_true',
                         help='Enable debug messages')
-    parser.add_argument('--show-ref',
-                        action='store_true',
-                        help='Show question references')
     parser.add_argument('--review',
                         action='store_true',
                         help='Review mode: doesn\'t generate test')
@@ -63,56 +56,50 @@ def parse_cmdline_arguments():
                         help='Show version information and exit')
     return parser.parse_args()
  
-
 # ----------------------------------------------------------------------------
 def get_logger_config(debug=False) -> dict:
     '''
     Load logger configuration from ~/.config directory if exists,
     otherwise set default paramenters.
     '''
+
+    file = 'logger-debug.yaml' if debug else 'logger.yaml'
+    path = os.path.expanduser(os.environ.get('XDG_CONFIG_HOME', '~/.config/'))
+    try:
+        return load_yaml(os.path.join(path, APP_NAME, file))
+    except OSError:
+        print('Using default logger configuration...')
+
     if debug:
-        filename = 'logger-debug.yaml'
         level = 'DEBUG'
+        fmt = '%(asctime)s %(levelname)-8s %(module)-12s%(lineno)4d| %(message)s'
+        dateformat = ''
     else:
-        filename = 'logger.yaml'
         level = 'INFO'
-
-    config_dir = environ.get('XDG_CONFIG_HOME', '~/.config/')
-    config_file = path.join(path.expanduser(config_dir), APP_NAME, filename)
-
-    default_config = {
-        'version': 1,
-        'formatters': {
-            'standard': {
-                'format': '%(asctime)s %(levelname)-8s %(message)s',
-                'datefmt': '%H:%M:%S',
-                },
-            },
-        'handlers': {
-            'default': {
-                'level': level,
-                'class': 'logging.StreamHandler',
-                'formatter': 'standard',
-                'stream': 'ext://sys.stdout',
+        fmt = '%(asctime)s| %(levelname)-8s| %(message)s'
+        dateformat = '%Y-%m-%d %H:%M:%S'
+    modules = ['main', 'serve', 'app', 'models', 'questions', 'test',
+               'testfactory', 'tools']
+    logger = {'handlers': ['default'], 'level': level, 'propagate': False}
+    return {
+            'version': 1,
+            'formatters': {
+                'standard': {
+                    'format': fmt,
+                    'datefmt': dateformat,
+                    },
                 },
-            },
-        'loggers': {
-            '': {  # configuration for serve.py
-                'handlers': ['default'],
-                'level': level,
+            'handlers': {
+                'default': {
+                    'level': level,
+                    'class': 'logging.StreamHandler',
+                    'formatter': 'standard',
+                    'stream': 'ext://sys.stdout',
+                    },
                 },
-            },
+            'loggers': {f'{APP_NAME}.{module}': logger for module in modules}
         }
  
-    modules = ['app', 'models', 'questions', 'test', 'testfactory', 'tools']
-    logger = {'handlers': ['default'], 'level': level, 'propagate': False}
-
-    default_config['loggers'].update({f'{APP_NAME}.{module}': logger
-                                      for module in modules})
-
-    return load_yaml(config_file, default=default_config)
-
-
 # ----------------------------------------------------------------------------
 def main() -> None:
     '''
@@ -122,15 +109,16 @@ def main() -&gt; None:
  
     # --- Setup logging ------------------------------------------------------
     logging.config.dictConfig(get_logger_config(args.debug))
-    logging.info('====================== Start Logging ======================')
+    logger = logging.getLogger(__name__)
+
+    logger.info('================== Start Logging ==================')
  
     # --- start application --------------------------------------------------
     config = {
         'testfile': args.testfile,
-        'debug': args.debug,
         'allow_all': args.allow_all,
         'allow_list': args.allow_list,
-        'show_ref': args.show_ref,
+        'debug': args.debug,
         'review': args.review,
         'correct': args.correct,
         }
@@ -138,24 +126,24 @@ def main() -&gt; None:
     try:
         app = App(config)
     except AppException:
-        logging.critical('Failed to start application.')
-        sys.exit(-1)
+        logger.critical('Failed to start application!')
+        sys.exit(1)
  
     # --- get SSL certificates -----------------------------------------------
     if 'XDG_DATA_HOME' in os.environ:
-        certs_dir = path.join(os.environ['XDG_DATA_HOME'], 'certs')
+        certs_dir = os.path.join(os.environ['XDG_DATA_HOME'], 'certs')
     else:
-        certs_dir = path.expanduser('~/.local/share/certs')
+        certs_dir = os.path.expanduser('~/.local/share/certs')
  
     ssl_opt = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
     try:
-        ssl_opt.load_cert_chain(path.join(certs_dir, 'cert.pem'),
-                                path.join(certs_dir, 'privkey.pem'))
+        ssl_opt.load_cert_chain(os.path.join(certs_dir, 'cert.pem'),
+                                os.path.join(certs_dir, 'privkey.pem'))
     except FileNotFoundError:
-        logging.critical('SSL certificates missing in %s', certs_dir)
-        sys.exit(-1)
+        logger.critical('SSL certificates missing in %s', certs_dir)
+        sys.exit(1)
  
-    # --- run webserver ----------------------------------------------------
+    # --- run webserver ------------------------------------------------------
     run_webserver(app=app, ssl_opt=ssl_opt, port=args.port, debug=args.debug)
  
  
 '''
+perguntations/models.py
 SQLAlchemy ORM
-
-The classes below correspond to database tables
 '''
  
+from typing import Any
  
 from sqlalchemy import Column, ForeignKey, Integer, Float, String
-from sqlalchemy.ext.declarative import declarative_base
-from sqlalchemy.orm import relationship
+from sqlalchemy.orm import declarative_base, relationship
  
  
-# ============================================================================
-#   Declare ORM
-Base = declarative_base()
+# FIXME Any is a workaround for static type checking
+# (https://github.com/python/mypy/issues/6372)
+Base: Any = declarative_base()
  
  
 # ----------------------------------------------------------------------------
@@ -26,11 +25,11 @@ class Student(Base):
     # ---
     tests = relationship('Test', back_populates='student')
  
-    def __str__(self):
-        return (f'Student:\n'
-                f'  id:         "{self.id}"\n'
-                f'  name:       "{self.name}"\n'
-                f'  password:   "{self.password}"\n')
+    def __repr__(self):
+        return (f'Student('
+            f'id={self.id!r}, '
+            f'name={self.name!r}, '
+            f'password={self.password!r})')
  
  
 # ----------------------------------------------------------------------------
@@ -52,18 +51,18 @@ class Test(Base):
     student = relationship('Student', back_populates='tests')
     questions = relationship('Question', back_populates='test')
  
-    def __str__(self):
-        return (f'Test:\n'
-                f'  id:         {self.id}\n'
-                f'  ref:        "{self.ref}"\n'
-                f'  title:      "{self.title}"\n'
-                f'  grade:      {self.grade}\n'
-                f'  state:      "{self.state}"\n'
-                f'  comment:    "{self.comment}"\n'
-                f'  starttime:  "{self.starttime}"\n'
-                f'  finishtime: "{self.finishtime}"\n'
-                f'  filename:   "{self.filename}"\n'
-                f'  student_id: "{self.student_id}"\n')
+    def __repr__(self):
+        return (f'Test('
+                f'id={self.id!r}, '
+                f'ref={self.ref!r}, '
+                f'title={self.title!r}, '
+                f'grade={self.grade!r}, '
+                f'state={self.state!r}, '
+                f'comment={self.comment!r}, '
+                f'starttime={self.starttime!r}, '
+                f'finishtime={self.finishtime!r}, '
+                f'filename={self.filename!r}, '
+                f'student_id={self.student_id!r})')
  
  
 # ---------------------------------------------------------------------------
@@ -82,13 +81,13 @@ class Question(Base):
     # ---
     test = relationship('Test', back_populates='questions')
  
-    def __str__(self):
-        return (f'Question:\n'
-                f'  id:         {self.id}\n'
-                f'  number:     {self.number}\n'
-                f'  ref:        "{self.ref}"\n'
-                f'  grade:      {self.grade}\n'
-                f'  comment:    "{self.comment}"\n'
-                f'  starttime:  "{self.starttime}"\n'
-                f'  finishtime: "{self.finishtime}"\n'
-                f'  test_id:    "{self.test_id}"\n')
+    def __repr__(self):
+        return (f'Question('
+                f'id={self.id!r}, '
+                f'number={self.number!r}, '
+                f'ref={self.ref!r}, '
+                f'grade={self.grade!r}, '
+                f'comment={self.comment!r}, '
+                f'starttime={self.starttime!r}, '
+                f'finishtime={self.finishtime!r}, '
+                f'test_id={self.test_id!r})')
@@ -137,9 +137,9 @@ class HighlightRenderer(mistune.Renderer):
         return '<table class="table table-sm"><thead class="thead-light">' \
                + header + '</thead><tbody>' + body + '</tbody></table>'
  
-    def image(self, src, title, alt):
+    def image(self, src, title, text):
         '''render image'''
-        alt = mistune.escape(alt, quote=True)
+        alt = mistune.escape(text, quote=True)
         if title is not None:
             if title:  # not empty string, show as caption
                 title = mistune.escape(title, quote=True)
 '''
-Classes the implement several types of questions.
+File:           perguntations/questions.py
+Description:    Classes the implement several types of questions.
 '''
  
  
 # python standard library
-import asyncio
 from datetime import datetime
 import logging
-from os import path
+import os
 import random
 import re
 from typing import Any, Dict, NewType
 import uuid
  
-
-# from urllib.error import HTTPError
-# import json
-# import http.client
-
-
 # this project
-from perguntations.tools import run_script, run_script_async
+from .tools import run_script, run_script_async
  
 # setup logger for this module
 logger = logging.getLogger(__name__)
  
-
 QDict = NewType('QDict', Dict[str, Any])
  
  
-
-
 class QuestionException(Exception):
     '''Exceptions raised in this module'''
  
@@ -45,8 +36,6 @@ class Question(dict):
     for each student.
     Instances can shuffle options or automatically generate questions.
     '''
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
  
     def gen(self) -> None:
         '''
@@ -96,9 +85,6 @@ class QuestionRadio(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         '''
         Sets defaults, performs checks and generates the actual question
@@ -128,8 +114,7 @@ class QuestionRadio(Question):
         # e.g.  correct: 2 --> correct: [0,0,1,0,0]
         if isinstance(self['correct'], int):
             if not 0 <= self['correct'] < nopts:
-                msg = (f'`correct` out of range 0..{nopts-1}. '
-                       f'In question "{self["ref"]}"')
+                msg = f'"{self["ref"]}": correct out of range 0..{nopts-1}'
                 logger.error(msg)
                 raise QuestionException(msg)
  
@@ -139,8 +124,7 @@ class QuestionRadio(Question):
         elif isinstance(self['correct'], list):
             # must match number of options
             if len(self['correct']) != nopts:
-                msg = (f'{nopts} options vs {len(self["correct"])} correct. '
-                       f'In question "{self["ref"]}"')
+                msg = f'"{self["ref"]}": number of options/correct mismatch'
                 logger.error(msg)
                 raise QuestionException(msg)
  
@@ -148,23 +132,20 @@ class QuestionRadio(Question):
             try:
                 self['correct'] = [float(x) for x in self['correct']]
             except (ValueError, TypeError) as exc:
-                msg = ('`correct` must be list of numbers or booleans.'
-                       f'In "{self["ref"]}"')
+                msg = f'"{self["ref"]}": correct must contain floats or bools'
                 logger.error(msg)
                 raise QuestionException(msg) from exc
  
             # check grade boundaries
             if self['discount'] and not all(0.0 <= x <= 1.0
                                             for x in self['correct']):
-                msg = ('`correct` values must be in the interval [0.0, 1.0]. '
-                       f'In "{self["ref"]}"')
+                msg = f'"{self["ref"]}": correct must be in [0.0, 1.0]'
                 logger.error(msg)
                 raise QuestionException(msg)
  
             # at least one correct option
             if all(x < 1.0 for x in self['correct']):
-                msg = ('At least one correct option is required. '
-                       f'In "{self["ref"]}"')
+                msg = f'"{self["ref"]}": has no correct options'
                 logger.error(msg)
                 raise QuestionException(msg)
  
@@ -231,9 +212,6 @@ class QuestionCheckbox(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
  
@@ -288,19 +266,6 @@ class QuestionCheckbox(Question):
                    f'Please fix "{self["ref"]}" in "{self["path"]}"')
             logger.error(msg)
             raise QuestionException(msg)
-            # msg0 = ('+--------------- BEHAVIOR CHANGE NOTICE ---------------+')
-            # msg1 = ('| Correct values in checkbox questions must be in the  |')
-            # msg2 = ('| interval [0.0, 1.0]. I will convert to the new       |')
-            # msg3 = ('| behavior, for now, but you should fix it.            |')
-            # msg4 = ('+------------------------------------------------------+')
-            # logger.warning(msg0)
-            # logger.warning(msg1)
-            # logger.warning(msg2)
-            # logger.warning(msg3)
-            # logger.warning(msg4)
-            # logger.warning('please fix "%s"', self["ref"])
-            # # normalize to [0,1]
-            # self['correct'] = [(x+1)/2 for x in self['correct']]
  
         # if an option is a list of (right, wrong), pick one
         options = []
@@ -356,9 +321,6 @@ class QuestionText(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
         self.set_defaults(QDict({
@@ -389,12 +351,13 @@ class QuestionText(Question):
     def transform(self, ans):
         '''apply optional filters to the answer'''
  
+        # apply transformations in sequence
         for transform in self['transform']:
             if transform == 'remove_space':  # removes all spaces
                 ans = ans.replace(' ', '')
             elif transform == 'trim':        # removes spaces around
                 ans = ans.strip()
-            elif transform == 'normalize_space':  # replaces multiple spaces by one
+            elif transform == 'normalize_space':  # replaces many spaces by one
                 ans = re.sub(r'\s+', ' ', ans.strip())
             elif transform == 'lower':       # convert to lowercase
                 ans = ans.lower()
@@ -410,7 +373,7 @@ class QuestionText(Question):
         super().correct()
  
         if self['answer'] is not None:
-            answer = self.transform(self['answer'])  # apply transformations
+            answer = self.transform(self['answer'])
             self['grade'] = 1.0 if answer in self['correct'] else 0.0
  
  
@@ -427,9 +390,6 @@ class QuestionTextRegex(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
  
@@ -442,14 +402,6 @@ class QuestionTextRegex(Question):
         if not isinstance(self['correct'], list):
             self['correct'] = [self['correct']]
  
-        # converts patterns to compiled versions
-        # try:
-        #     self['correct'] = [re.compile(a) for a in self['correct']]
-        # except Exception as exc:
-        #     msg = f'Failed to compile regex in "{self["ref"]}"'
-        #     logger.error(msg)
-        #     raise QuestionException(msg) from exc
-
     # ------------------------------------------------------------------------
     def correct(self) -> None:
         super().correct()
@@ -464,15 +416,6 @@ class QuestionTextRegex(Question):
                                  regex, self['answer'])
             self['grade'] = 0.0
  
-                # try:
-                #     if regex.match(self['answer']):
-                #         self['grade'] = 1.0
-                #         return
-                # except TypeError:
-                #     logger.error('While matching regex %s with answer "%s".',
-                #                  regex.pattern, self["answer"])
-
-
 # ============================================================================
 class QuestionNumericInterval(Question):
     '''An instance of QuestionTextNumeric will always have the keys:
@@ -484,9 +427,6 @@ class QuestionNumericInterval(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
  
@@ -548,9 +488,6 @@ class QuestionTextArea(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
  
@@ -561,7 +498,7 @@ class QuestionTextArea(Question):
             'args': []
             }))
  
-        self['correct'] = path.join(self['path'], self['correct'])
+        self['correct'] = os.path.join(self['path'], self['correct'])
  
     # ------------------------------------------------------------------------
     def correct(self) -> None:
@@ -625,129 +562,6 @@ class QuestionTextArea(Question):
  
  
 # ============================================================================
-# class QuestionCode(Question):
-#     '''
-#     Submits answer to a JOBE server to compile and run against the test cases.
-#     '''
-
-#     _outcomes = {
-#         0:  'JOBE outcome: Successful run',
-#         11: 'JOBE outcome: Compile error',
-#         12: 'JOBE outcome: Runtime error',
-#         13: 'JOBE outcome: Time limit exceeded',
-#         15: 'JOBE outcome: Successful run',
-#         17: 'JOBE outcome: Memory limit exceeded',
-#         19: 'JOBE outcome: Illegal system call',
-#         20: 'JOBE outcome: Internal error, please report',
-#         21: 'JOBE outcome: Server overload',
-#         }
-
-#     # ------------------------------------------------------------------------
-#     def __init__(self, q: QDict) -> None:
-#         super().__init__(q)
-
-#         self.set_defaults(QDict({
-#             'text': '',
-#             'timeout': 5,   # seconds
-#             'server': '127.0.0.1',  # JOBE server
-#             'language': 'c',
-#             'correct': [{'stdin': '', 'stdout': '', 'stderr': '', 'args': ''}],
-#             }))
-
-    # ------------------------------------------------------------------------
-    # def correct(self) -> None:
-    #     super().correct()
-
-    #     if self['answer'] is None:
-    #         return
-
-    #     # submit answer to JOBE server
-    #     resource = '/jobe/index.php/restapi/runs/'
-    #     headers = {"Content-type": "application/json; charset=utf-8",
-    #                "Accept": "application/json"}
-
-    #     for expected in self['correct']:
-    #         data_json = json.dumps({
-    #             'run_spec' : {
-    #                 'language_id': self['language'],
-    #                 'sourcecode': self['answer'],
-    #                 'input': expected.get('stdin', ''),
-    #                  },
-    #             })
-
-    #         try:
-    #             connect = http.client.HTTPConnection(self['server'])
-    #             connect.request(
-    #                 method='POST',
-    #                 url=resource,
-    #                 body=data_json,
-    #                 headers=headers
-    #                 )
-    #             response = connect.getresponse()
-    #             logger.debug('JOBE response status %d', response.status)
-    #             if response.status != 204:
-    #                 content = response.read().decode('utf8')
-    #                 if content:
-    #                     result = json.loads(content)
-    #             connect.close()
-
-    #         except (HTTPError, ValueError):
-    #             logger.error('HTTPError while connecting to JOBE server')
-
-    #         try:
-    #             outcome = result['outcome']
-    #         except (NameError, TypeError, KeyError):
-    #             logger.error('Bad result returned from JOBE server: %s', result)
-    #             return
-    #         logger.debug(self._outcomes[outcome])
-
-
-
-    #         if result['cmpinfo']:  # compiler errors and warnings
-    #             self['comments'] = f'Erros de compilação:\n{result["cmpinfo"]}'
-    #             self['grade'] = 0.0
-    #             return
-
-    #         if result['stdout'] != expected.get('stdout', ''):
-    #             self['comments'] = 'O output gerado é diferente do esperado.'  # FIXME mostrar porque?
-    #             self['grade'] = 0.0
-    #             return
-
-    #     self['comments'] = 'Ok!'
-    #     self['grade'] = 1.0
-
-
-    # # ------------------------------------------------------------------------
-    # async def correct_async(self) -> None:
-    #     self.correct()  # FIXME there is no async correction!!!
-
-
-            # out = run_script(
-            #     script=self['correct'],
-            #     args=self['args'],
-            #     stdin=self['answer'],
-            #     timeout=self['timeout']
-            #     )
-
-            # if out is None:
-            #     logger.warning('No grade after running "%s".', self["correct"])
-            #     self['comments'] = 'O programa de correcção abortou...'
-            #     self['grade'] = 0.0
-            # elif isinstance(out, dict):
-            #     self['comments'] = out.get('comments', '')
-            #     try:
-            #         self['grade'] = float(out['grade'])
-            #     except ValueError:
-            #         logger.error('Output error in "%s".', self["correct"])
-            #     except KeyError:
-            #         logger.error('No grade in "%s".', self["correct"])
-            # else:
-            #     try:
-            #         self['grade'] = float(out)
-            #     except (TypeError, ValueError):
-            #         logger.error('Invalid grade in "%s".', self["correct"])
-
-# ============================================================================
 class QuestionInformation(Question):
     '''
     Not really a question, just an information panel.
@@ -755,9 +569,6 @@ class QuestionInformation(Question):
     '''
  
     # ------------------------------------------------------------------------
-    # def __init__(self, q: QDict) -> None:
-    #     super().__init__(q)
-
     def gen(self) -> None:
         super().gen()
         self.set_defaults(QDict({
@@ -770,7 +581,6 @@ class QuestionInformation(Question):
         self['grade'] = 1.0  # always "correct" but points should be zero!
  
  
-
 # ============================================================================
 def question_from(qdict: QDict) -> Question:
     '''
@@ -783,7 +593,6 @@ def question_from(qdict: QDict) -&gt; Question:
         'text-regex':       QuestionTextRegex,
         'numeric-interval': QuestionNumericInterval,
         'textarea':         QuestionTextArea,
-        # 'code':             QuestionCode,
         # -- informative panels --
         'information':      QuestionInformation,
         'success':          QuestionInformation,
@@ -856,17 +665,17 @@ class QFactory():
         logger.debug('generating %s...', self.qdict["ref"])
         # Shallow copy so that script generated questions will not replace
         # the original generators
-        qdict = self.qdict.copy()
+        qdict = QDict(self.qdict.copy())
         qdict['qid'] = str(uuid.uuid4())  # unique for each question
  
         # If question is of generator type, an external program will be run
         # which will print a valid question in yaml format to stdout. This
         # output is then yaml parsed into a dictionary `q`.
         if qdict['type'] == 'generator':
-            logger.debug(' \\_ Running "%s".', qdict['script'])
+            logger.debug(' \\_ Running "%s"', qdict['script'])
             qdict.setdefault('args', [])
             qdict.setdefault('stdin', '')
-            script = path.join(qdict['path'], qdict['script'])
+            script = os.path.join(qdict['path'], qdict['script'])
             out = await run_script_async(script=script,
                                          args=qdict['args'],
                                          stdin=qdict['stdin'])
@@ -875,8 +684,3 @@ class QFactory():
         question = question_from(qdict)  # returns a Question instance
         question.gen()
         return question
-
-    # ------------------------------------------------------------------------
-    def generate(self) -> Question:
-        '''generate question (synchronous version)'''
-        return asyncio.get_event_loop().run_until_complete(self.gen_async())
@@ -10,23 +10,27 @@ import asyncio
 import base64
 import functools
 import json
-import logging.config
+import logging
 import mimetypes
 from os import path
 import re
 import signal
 import sys
 from timeit import default_timer as timer
+from typing import Dict, Tuple
 import uuid
  
 # user installed libraries
 import tornado.ioloop
 import tornado.web
-# import tornado.websocket
 import tornado.httpserver
  
 # this project
-from perguntations.parser_markdown import md_to_html
+from .parser_markdown import md_to_html
+
+
+# setup logger for this module
+logger = logging.getLogger(__name__)
  
  
 # ----------------------------------------------------------------------------
@@ -41,8 +45,6 @@ class WebApplication(tornado.web.Application):
             (r'/review', ReviewHandler),
             (r'/admin', AdminHandler),
             (r'/file', FileHandler),
-            # (r'/root', MainHandler),
-            # (r'/ws', AdminSocketHandler),
             (r'/adminwebservice', AdminWebservice),
             (r'/studentwebservice', StudentWebservice),
             (r'/', RootHandler),
@@ -64,11 +66,10 @@ class WebApplication(tornado.web.Application):
 # ----------------------------------------------------------------------------
 def admin_only(func):
     '''
-    Decorator used to restrict access to the administrator.
-    Example:
+    Decorator to restrict access to the administrator:
  
-        @admin_only
-        def get(self): ...
+    @admin_only
+    def get(self):
     '''
     @functools.wraps(func)
     async def wrapper(self, *args, **kwargs):
@@ -92,6 +93,11 @@ class BaseHandler(tornado.web.RequestHandler):
         '''simplifies access to the application a little bit'''
         return self.application.testapp
  
+    # @property
+    # def debug(self) -> bool:
+    #     '''check if is running in debug mode'''
+    #     return self.application.testapp.debug
+
     def get_current_user(self):
         '''
         Since HTTP is stateless, a cookie is used to identify the user.
@@ -104,72 +110,15 @@ class BaseHandler(tornado.web.RequestHandler):
  
  
 # ----------------------------------------------------------------------------
-# class MainHandler(BaseHandler):
-
-#     @tornado.web.authenticated
-#     @admin_only
-#     def get(self):
-#         self.render("admin-ws.html", students=self.testapp.get_students_state())
-
-
-# # ----------------------------------------------------------------------------
-# class AdminSocketHandler(tornado.websocket.WebSocketHandler):
-#     waiters = set()
-#     # cache = []
-
-#     # def get_compression_options(self):
-#     #     return {}   # Non-None enables compression with default options.
-
-#     # called when opening connection
-#     def open(self):
-#         logging.debug('[AdminSocketHandler.open]')
-#         AdminSocketHandler.waiters.add(self)
-
-#     # called when closing connection
-#     def on_close(self):
-#         logging.debug('[AdminSocketHandler.on_close]')
-#         AdminSocketHandler.waiters.remove(self)
-
-#     # @classmethod
-#     # def update_cache(cls, chat):
-#     #     logging.debug(f'[AdminSocketHandler.update_cache] "{chat}"')
-#     #     cls.cache.append(chat)
-
-#     # @classmethod
-#     # def send_updates(cls, chat):
-#     #     logging.info("sending message to %d waiters", len(cls.waiters))
-#     #     for waiter in cls.waiters:
-#     #         try:
-#     #             waiter.write_message(chat)
-#     #         except Exception:
-#     #             logging.error("Error sending message", exc_info=True)
-
-#     # handle incomming messages
-#     def on_message(self, message):
-#         logging.info(f"[AdminSocketHandler.onmessage] got message {message}")
-#         parsed = tornado.escape.json_decode(message)
-#         print(parsed)
-#         chat = {"id": str(uuid.uuid4()), "body": parsed["body"]}
-#         print(chat)
-#         chat["html"] = tornado.escape.to_basestring(
-#             '<div>' + chat['body'] + '</div>'
-#             # self.render_string("message.html", message=chat)
-#         )
-#         print(chat)
-
-#         AdminSocketHandler.update_cache(chat)  # store msgs
-#         AdminSocketHandler.send_updates(chat)  # send to clients
-
-# ----------------------------------------------------------------------------
 # pylint: disable=abstract-method
 class LoginHandler(BaseHandler):
     '''Handles /login'''
  
     _prefix = re.compile(r'[a-z]')
     _error_msg = {
-        'wrong_password': 'Password errada',
-        'already_online': 'Já está online, não pode entrar duas vezes',
-        'unauthorized': 'Não está autorizado a fazer o teste'
+        'wrong_password': 'Senha errada',
+        'not_allowed': 'Não está autorizado a fazer o teste',
+        'nonexistent': 'Número de aluno inválido'
     }
  
     def get(self):
@@ -178,7 +127,7 @@ class LoginHandler(BaseHandler):
  
     async def post(self):
         '''Authenticates student and login.'''
-        uid = self._prefix.sub('', self.get_body_argument('uid'))
+        uid = self.get_body_argument('uid')
         password = self.get_body_argument('pw')
         headers = {
             'remote_ip': self.request.remote_ip,
@@ -187,7 +136,7 @@ class LoginHandler(BaseHandler):
  
         error = await self.testapp.login(uid, password, headers)
  
-        if error:
+        if error is not None:
             await asyncio.sleep(3)  # delay to avoid spamming the server...
             self.render('login.html', error=self._error_msg[error])
         else:
@@ -203,8 +152,8 @@ class LogoutHandler(BaseHandler):
     @tornado.web.authenticated
     def get(self):
         '''Logs out a user.'''
-        self.clear_cookie('perguntations_user')
         self.testapp.logout(self.current_user)
+        self.clear_cookie('perguntations_user')
         self.render('login.html', error='')
  
  
@@ -214,7 +163,7 @@ class LogoutHandler(BaseHandler):
 # pylint: disable=abstract-method
 class RootHandler(BaseHandler):
     '''
-    Generates test to student.
+    Presents test to student.
     Receives answers, corrects the test and sends back the grade.
     Redirects user 0 to /admin.
     '''
@@ -227,7 +176,6 @@ class RootHandler(BaseHandler):
         'text-regex':   'question-text.html',
         'numeric-interval': 'question-text.html',
         'textarea':     'question-textarea.html',
-        'code':         'question-textarea.html',
         # -- information panels --
         'information':  'question-information.html',
         'success':      'question-information.html',
@@ -243,77 +191,67 @@ class RootHandler(BaseHandler):
         Sends test to student or redirects 0 to admin page.
         Multiple calls to this function will return the same test.
         '''
-
         uid = self.current_user
-        logging.debug('"%s" GET /', uid)
+        logger.debug('"%s" GET /', uid)
  
         if uid == '0':
             self.redirect('/admin')
-            return
-
-        test = await self.testapp.get_test_or_generate(uid)
-        self.render('test.html', t=test, md=md_to_html, templ=self._templates)
-
+        else:
+            test = self.testapp.get_test(uid)
+            name = self.testapp.get_name(uid)
+            self.render('test.html', t=test, uid=uid, name=name, md=md_to_html,
+                        templ=self._templates, debug=self.testapp.debug)
  
     # --- POST
     @tornado.web.authenticated
     async def post(self):
         '''
         Receives answers, fixes some html weirdness, corrects test and
-        sends back the grade.
+        renders the grade.
  
         self.request.arguments = {'answered-0': [b'on'], '0': [b'13.45']}
-        builds dictionary ans={0: 'answer0', 1:, 'answer1', ...}
-        unanswered questions not included.
+        builds dictionary ans = {0: 'answer0', 1:, 'answer1', ...}
+        unanswered questions are not included.
         '''
-        timeit_start = timer()  # performance timer
+        starttime = timer()  # performance timer
  
         uid = self.current_user
-        logging.debug('"%s" POST /', uid)
+        logger.debug('"%s" POST /', uid)
  
-        try:
-            test = self.testapp.get_test(uid)
-        except KeyError as exc:
-            logging.warning('"%s" POST / raised 403 Forbidden', uid)
-            raise tornado.web.HTTPError(403) from exc # Forbidden
+        test = self.testapp.get_test(uid)
+        if test is None:
+            logger.warning('"%s" submitted but no test running - Err 403', uid)
+            raise tornado.web.HTTPError(403)  # Forbidden
  
         ans = {}
         for i, question in enumerate(test['questions']):
             qid = str(i)
-            if 'answered-' + qid in self.request.arguments:
+            if f'answered-{qid}' in self.request.arguments:
                 ans[i] = self.get_body_arguments(qid)
  
                 # remove enclosing list in some question types
                 if question['type'] == 'radio':
-                    if not ans[i]:
-                        ans[i] = None
-                    else:
-                        ans[i] = ans[i][0]
+                    ans[i] = ans[i][0] if ans[i] else None
                 elif question['type'] in ('text', 'text-regex', 'textarea',
-                                          'numeric-interval', 'code'):
+                                          'numeric-interval'):
                     ans[i] = ans[i][0]
  
         # submit answered questions, correct
         await self.testapp.submit_test(uid, ans)
  
-        # show final grade and grades of other tests in the database
-        # allgrades = self.testapp.get_student_grades_from_all_tests(uid)
-        # grade = self.testapp.get_student_grade(uid)
-
-        self.render('grade.html', t=test)
+        name = self.testapp.get_name(uid)
+        self.render('grade.html', t=test, uid=uid, name=name)
         self.clear_cookie('perguntations_user')
         self.testapp.logout(uid)
-
-        timeit_finish = timer()
-        logging.info('  elapsed time: %fs', timeit_finish-timeit_start)
+        logger.info('  elapsed time: %fs', timer() - starttime)
  
  
 # ----------------------------------------------------------------------------
 # pylint: disable=abstract-method
 class StudentWebservice(BaseHandler):
     '''
-    Receive ajax from students in the test in response from focus, unfocus and
-    resize events.
+    Receive ajax from students during the test in response to the events
+    focus, unfocus and resize.
     '''
  
     @tornado.web.authenticated
@@ -321,8 +259,9 @@ class StudentWebservice(BaseHandler):
         '''handle ajax post'''
         uid = self.current_user
         cmd = self.get_body_argument('cmd', None)
-        value = json.loads(self.get_body_argument('value', None))
-        self.testapp.event_test(uid, cmd, value)
+        value = self.get_body_argument('value', None)
+        if cmd is not None and value is not None:
+            self.testapp.register_event(uid, cmd, json.loads(value))
  
  
 # ----------------------------------------------------------------------------
@@ -337,16 +276,17 @@ class AdminWebservice(BaseHandler):
     async def get(self):
         '''admin webservices that do not change state'''
         cmd = self.get_query_argument('cmd')
+        logger.debug('GET /adminwebservice %s', cmd)
+
         if cmd == 'testcsv':
-            test_ref, data = self.testapp.get_test_csv()
+            test_ref, data = self.testapp.get_grades_csv()
             self.set_header('Content-Type', 'text/csv')
             self.set_header('content-Disposition',
                             f'attachment; filename={test_ref}.csv')
             self.write(data)
             await self.flush()
-
-        if cmd == 'questionscsv':
-            test_ref, data = self.testapp.get_questions_csv()
+        elif cmd == 'questionscsv':
+            test_ref, data = self.testapp.get_detailed_grades_csv()
             self.set_header('Content-Type', 'text/csv')
             self.set_header('content-Disposition',
                             f'attachment; filename={test_ref}-detailed.csv')
@@ -359,6 +299,7 @@ class AdminWebservice(BaseHandler):
 class AdminHandler(BaseHandler):
     '''Handle /admin'''
  
+    # --- GET
     @tornado.web.authenticated
     @admin_only
     async def get(self):
@@ -366,24 +307,18 @@ class AdminHandler(BaseHandler):
         Admin page.
         '''
         cmd = self.get_query_argument('cmd', default=None)
+        logger.debug('GET /admin (cmd=%s)', cmd)
  
-        if cmd == 'students_table':
-            data = {'data': self.testapp.get_students_state()}
-            self.write(json.dumps(data, default=str))
+        if cmd is None:
+            self.render('admin.html')
         elif cmd == 'test':
-            data = {
-                'data': {
-                    'title': self.testapp.testfactory['title'],
-                    'ref': self.testapp.testfactory['ref'],
-                    'filename': self.testapp.testfactory['testfile'],
-                    'database': self.testapp.testfactory['database'],
-                    'answers_dir': self.testapp.testfactory['answers_dir'],
-                    }
-                }
+            data = { 'data': self.testapp.get_test_config() }
+            self.write(json.dumps(data, default=str))
+        elif cmd == 'students_table':
+            data = {'data': self.testapp.get_students_state()}
             self.write(json.dumps(data, default=str))
-        else:
-            self.render('admin.html')
  
+    # --- POST
     @tornado.web.authenticated
     @admin_only
     async def post(self):
@@ -392,6 +327,7 @@ class AdminHandler(BaseHandler):
         '''
         cmd = self.get_body_argument('cmd', None)
         value = self.get_body_argument('value', None)
+        logger.debug('POST /admin (cmd=%s, value=%s)')
  
         if cmd == 'allow':
             self.testapp.allow_student(value)
@@ -402,15 +338,11 @@ class AdminHandler(BaseHandler):
         elif cmd == 'deny_all':
             self.testapp.deny_all_students()
         elif cmd == 'reset_password':
-            await self.testapp.update_student_password(uid=value, password='')
-
-        elif cmd == 'insert_student':
+            await self.testapp.set_password(uid=value, pw='')
+        elif cmd == 'insert_student' and value is not None:
             student = json.loads(value)
-            self.testapp.insert_new_student(uid=student['number'],
-                                            name=student['name'])
-
-        else:
-            logging.error('Unknown command: "%s"', cmd)
+            await self.testapp.insert_new_student(uid=student['number'],
+                                                  name=student['name'])
  
  
 # ----------------------------------------------------------------------------
@@ -422,6 +354,7 @@ class FileHandler(BaseHandler):
     Handles static files from questions like images, etc.
     '''
  
+    _filecache: Dict[Tuple[str, str], bytes] = {}
  
     @tornado.web.authenticated
     async def get(self):
@@ -429,40 +362,47 @@ class FileHandler(BaseHandler):
         Returns requested file. Files are obtained from the 'public' directory
         of each question.
         '''
-
         uid = self.current_user
         ref = self.get_query_argument('ref', None)
         image = self.get_query_argument('image', None)
+        logger.debug('GET /file (ref=%s, image=%s)', ref, image)
+
+        if ref is None or image is None:
+            return
+
         content_type = mimetypes.guess_type(image)[0]
  
-        if uid != '0':
-            test = self.testapp.get_student_test(uid)
-        else:
-            logging.error('FIXME Cannot serve images for review.')
-            raise tornado.web.HTTPError(404)  # FIXME admin
+        if (ref, image) in self._filecache:
+            logger.debug('using cached file')
+            self.write(self._filecache[(ref, image)])
+            if content_type is not None:
+                self.set_header("Content-Type", content_type)
+            await self.flush()
+            return
  
-        if test is None:
-            raise tornado.web.HTTPError(404)  # Not Found
+        try:
+            test = self.testapp.get_test(uid)
+        except KeyError:
+            logger.warning('Could not get test to serve image file')
+            raise tornado.web.HTTPError(404) from None  # Not Found
  
+        # search for the question that contains the image
         for question in test['questions']:
-            # search for the question that contains the image
             if question['ref'] == ref:
                 filepath = path.join(question['path'], 'public', image)
+
                 try:
-                    file = open(filepath, 'rb')
-                except FileNotFoundError:
-                    logging.error('File not found: %s', filepath)
-                except PermissionError:
-                    logging.error('No permission: %s', filepath)
+                    with open(filepath, 'rb') as file:
+                        data = file.read()
                 except OSError:
-                    logging.error('Error opening file: %s', filepath)
-                else:
-                    data = file.read()
-                    file.close()
+                    logger.error('Error reading file "%s"', filepath)
+                    return
+                self._filecache[(ref, image)] = data
+                self.write(data)
+                if content_type is not None:
                     self.set_header("Content-Type", content_type)
-                    self.write(data)
-                    await self.flush()
-                break
+                await self.flush()
+                return
  
  
 # --- REVIEW -----------------------------------------------------------------
@@ -479,7 +419,6 @@ class ReviewHandler(BaseHandler):
         'text-regex':   'review-question-text.html',
         'numeric-interval': 'review-question-text.html',
         'textarea':     'review-question-text.html',
-        'code':         'review-question-text.html',
         # -- information panels --
         'information':  'review-question-information.html',
         'success':      'review-question-information.html',
@@ -494,26 +433,28 @@ class ReviewHandler(BaseHandler):
         Opens JSON file with a given corrected test and renders it
         '''
         test_id = self.get_query_argument('test_id', None)
-        logging.info('Review test %s.', test_id)
+        logger.info('Review test %s.', test_id)
         fname = self.testapp.get_json_filename_of_test(test_id)
  
         if fname is None:
             raise tornado.web.HTTPError(404)  # Not Found
  
         try:
-            with open(path.expanduser(fname)) as jsonfile:
+            with open(path.expanduser(fname), encoding='utf-8') as jsonfile:
                 test = json.load(jsonfile)
         except OSError:
             msg = f'Cannot open "{fname}" for review.'
-            logging.error(msg)
+            logger.error(msg)
             raise tornado.web.HTTPError(status_code=404, reason=msg) from None
         except json.JSONDecodeError as exc:
             msg = f'JSON error in "{fname}": {exc}'
-            logging.error(msg)
+            logger.error(msg)
             raise tornado.web.HTTPError(status_code=404, reason=msg)
  
-        self.render('review.html', t=test, md=md_to_html,
-                    templ=self._templates)
+        uid = test['student']
+        name = self.testapp.get_name(uid)
+        self.render('review.html', t=test, uid=uid, name=name, md=md_to_html,
+                    templ=self._templates, debug=self.testapp.debug)
  
  
 # ----------------------------------------------------------------------------
@@ -524,7 +465,7 @@ def signal_handler(*_):
     reply = input(' --> Stop webserver? (yes/no) ')
     if reply.lower() == 'yes':
         tornado.ioloop.IOLoop.current().stop()
-        logging.critical('Webserver stopped.')
+        logger.critical('Webserver stopped.')
         sys.exit(0)
  
 # ----------------------------------------------------------------------------
@@ -534,33 +475,33 @@ def run_webserver(app, ssl_opt, port, debug):
     '''
  
     # --- create web application
-    logging.info('-----------------------------------------------------------')
-    logging.info('Starting WebApplication (tornado)')
+    logger.info('-------- Starting WebApplication (tornado) --------')
     try:
         webapp = WebApplication(app, debug=debug)
     except Exception:
-        logging.critical('Failed to start web application.')
+        logger.critical('Failed to start web application.')
         raise
  
+    # --- create httpserver
     try:
         httpserver = tornado.httpserver.HTTPServer(webapp, ssl_options=ssl_opt)
     except ValueError:
-        logging.critical('Certificates cert.pem, privkey.pem not found')
+        logger.critical('Certificates cert.pem, privkey.pem not found')
         sys.exit(1)
  
     try:
         httpserver.listen(port)
     except OSError:
-        logging.critical('Cannot bind port %d. Already in use?', port)
+        logger.critical('Cannot bind port %d. Already in use?', port)
         sys.exit(1)
  
-    logging.info('Webserver listening on %d...  (Ctrl-C to stop)', port)
+    logger.info('Listening on port %d...  (Ctrl-C to stop)', port)
     signal.signal(signal.SIGINT, signal_handler)
  
     # --- run webserver
     try:
         tornado.ioloop.IOLoop.current().start()  # running...
     except Exception:
-        logging.critical('Webserver stopped!')
+        logger.critical('Webserver stopped!')
         tornado.ioloop.IOLoop.current().stop()
         raise
@@ -117,16 +117,13 @@ $(document).ready(function() {
                         d = json.data[i];
                         var uid = d['uid'];
                         var checked = d['allowed'] ? 'checked' : '';
-                        var password_defined = d['password_defined'] ? ' <span class="badge badge-secondary"><i class="fa fa-key" aria-hidden="true"></i></span>' : '';
+                        // var password_defined = d['password_defined'] ? ' <span class="badge badge-secondary"><i class="fa fa-key" aria-hidden="true"></i></span>' : '';
                         var hora_inicio = d['start_time'] ? ' <span class="badge badge-success"><i class="fas fa-hourglass-start"></i> ' + d['start_time'].slice(11,16) + '</span>': '';
                         var unfocus = d['unfocus'] ? ' <span class="badge badge-danger">unfocus</span>' : '';
-                        var area = '';
-                        if (d['start_time'] ) {
-                            if (d['area'] > 75)
-                                area = ' <span class="badge badge-success"><i class="fas fa-desktop"></i> ' + Math.round(d['area']) + '%</span>';
-                            else
-                                area = ' <span class="badge badge-danger"><i class="fas fa-desktop"></i> ' + Math.round(d['area']) + '%</span>';
-                        };
+                        if (d['area'] > 75)
+                            area = ' <span class="badge badge-success"><i class="fas fa-desktop"></i>' + Math.round(d['area']) + '%</span>';
+                        else
+                            area = ' <span class="badge badge-danger"><i class="fas fa-desktop"></i> ' + Math.round(d['area']) + '%</span>';
                         var g = d['grades'];
  
                         t[i] = [];
@@ -134,7 +131,7 @@ $(document).ready(function() {
                         t[i][1] = '<input type="checkbox" name="' + uid + '" value="true"' + checked + '> ';
                         t[i][2] = uid;
                         t[i][3] = d['name'];
-                        t[i][4] = password_defined + hora_inicio + area + unfocus;
+                        t[i][4] = d['online'] ? hora_inicio + area + unfocus : '';
  
                         var gbar = '';
                         for (var j=0; j < g.length; j++)
@@ -53,8 +53,8 @@
           Acções
         </a>
         <div class="dropdown-menu dropdown-menu-right" aria-labelledby="navbarDropdownAluno">
-          <a class="dropdown-item" href="#" id="novo_aluno" data-toggle="modal" data-target="#novo_aluno_modal">Inserir novo aluno...</a>
-          <a class="dropdown-item" href="#" id="reset_password_menu" data-toggle="modal" data-target="#reset_password_modal">Reset password do aluno...</a>
+          <a class="dropdown-item" href="#" id="novo_aluno" data-toggle="modal" data-target="#novo_aluno_modal">Novo aluno...</a>
+          <a class="dropdown-item" href="#" id="reset_password_menu" data-toggle="modal" data-target="#reset_password_modal">Limpar password...</a>
           <a class="dropdown-item" href="#" id="allow_all">Autorizar todos</a>
           <a class="dropdown-item" href="#" id="deny_all">Desautorizar todos</a>
           <div class="dropdown-divider"></div>
@@ -72,7 +72,7 @@
     <p>
       Referência: <code id="ref">--</code><br>
       Ficheiro de configuração do teste: <code id="filename">--</code><br>
-      Testes em formato JSON no directório: <code id="answers_dir">--</code><br>
+      Directório com os testes entregues: <code id="answers_dir">--</code><br>
       Base de dados: <code id="database">--</code><br>
     </p>
     <p>
@@ -31,8 +31,8 @@
     </ul>
     <span class="navbar-text">
       <i class="fas fa-user" aria-hidden="true"></i>
-      <span id="name">{{ escape(t['student']['name']) }}</span>
-      (<span id="number">{{ escape(t['student']['number']) }}</span>)
+      <span id="name">{{ escape(name) }}</span>
+      (<span id="number">{{ escape(uid) }}</span>)
       <span class="caret"></span>
     </span>
   </div>
@@ -17,9 +17,9 @@
     {{ md(q['text']) }}
   </div>
  
-  {% if show_ref %}
+  {% if debug %}
     <hr>
     file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
     ref:  <code>{{ q['ref'] }}</code>
   {% end %}
-</div>
 \ No newline at end of file
+</div>
@@ -29,11 +29,11 @@
     </p>
   </div>
  
-  {% if show_ref %}
+  {% if debug %}
     <div class="card-footer">
       file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
       ref:  <code>{{ q['ref'] }}</code>
     </div>
   {% end %}
 </div>
-{% end %}
 \ No newline at end of file
+{% end %}
@@ -16,9 +16,9 @@
 	<div id="text">
 		{{ md(q['text']) }}
 	</div>
-    {% if t['show_ref'] %}
+    {% if debug %}
         <hr>
         file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
         ref:  <code>{{ q['ref'] }}</code>
     {% end %}
-</div>
 \ No newline at end of file
+</div>
@@ -65,7 +65,7 @@
             {% end %}
           {% end %}
  
-          {% if t['show_ref'] %}
+          {% if debug %}
             <hr>
             file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
             ref:  <code>{{ q['ref'] }}</code>
@@ -109,7 +109,7 @@
           {% end %}
         </p>
  
-        {% if t['show_ref'] %}
+        {% if debug %}
           <hr>
           file: <code>{{ q['path'] }}/{{ q['filename'] }}</code><br>
           ref:  <code>{{ q['ref'] }}</code>
@@ -118,4 +118,4 @@
       </div> <!-- card-footer -->
     </div> <!-- card -->
   {% end %} <!-- if answer not None -->
-{% end %} <!-- block -->
 \ No newline at end of file
+{% end %} <!-- block -->
@@ -59,8 +59,8 @@
         <li class="nav-item">
           <span class="navbar-text">
             <i class="fas fa-user" aria-hidden="true"></i>
-            <span id="name">{{ escape(t['student']['name']) }}</span>
-            (<span id="number">{{ escape(t['student']['number']) }}</span>)
+            <span id="name">{{ escape(name) }}</span>
+            (<span id="number">{{ escape(uid) }}</span>)
             <span class="caret"></span>
           </span>
         </li>
@@ -97,9 +97,12 @@
       <div class="row">
         <label for="nota" class="col-sm-2">Nota:</label>
         <div class="col-sm-10" id="nota">
-          <span class="badge badge-primary">{{ round(t['grade'], 2) }}</span> valores
-          {% if t['state'] == 'QUIT' %}
-              (DESISTÊNCIA)
+          {% if t['state'] == 'CORRECTED' %}
+            <span class="badge badge-primary">{{ round(t['grade'], 2) }}</span> valores
+          {% elif t['state'] == 'SUBMITTED' %}
+            (não corrigido)
+          {% elif t['state'] == 'QUIT' %}
+            (DESISTIU)
           {% end %}
         </div>
       </div>
@@ -113,7 +116,7 @@
   </div> <!-- jumbotron -->
  
   {% for i, q in enumerate(t['questions']) %}
-    {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), t=t) %}
+    {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), t=t, debug=debug) %}
   {% end %}
  
 </div> <!-- container -->
@@ -74,8 +74,8 @@
         <li class="nav-item">
           <span class="navbar-text">
             <i class="fas fa-user" aria-hidden="true"></i>
-            <span id="name">{{ escape(t['student']['name']) }}</span>
-            (<span id="number">{{ escape(t['student']['number']) }}</span>)
+            <span id="name">{{ escape(name) }}</span>
+            (<span id="number">{{ escape(uid) }}</span>)
             <span class="caret"></span>
           </span>
         </li>
@@ -93,11 +93,11 @@
  
       <div class="row">
         <label for="nome" class="col-sm-3">Nome:</label>
-        <div class="col-sm-9" id="nome">{{ escape(t['student']['name']) }}</div>
+        <div class="col-sm-9" id="nome">{{ escape(name) }}</div>
       </div>
       <div class="row">
         <label for="numero" class="col-sm-3">Número:</label>
-        <div class="col-sm-9" id="numero">{{ escape(t['student']['number']) }}</div>
+        <div class="col-sm-9" id="numero">{{ escape(uid) }}</div>
       </div>
  
       <div class="row">
@@ -114,12 +114,14 @@
     {% module xsrf_form_html() %}
  
     {% for i, q in enumerate(t['questions']) %}
-      {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), show_ref=t['show_ref']) %}
+      {% module Template(templ[q['type']], i=i, q=q, md=md(q['ref']), debug=debug) %}
     {% end %}
  
     <div class="form-row">
       <div class="col-12">
-        <button type="button" class="btn btn-success btn-lg btn-block" data-toggle="modal" data-target="#confirmar" id="form-button-submit">Submeter teste</button>
+        <button type="button" class="btn btn-success btn-lg btn-block" data-toggle="modal" data-target="#confirmar" id="form-button-submit">
+          Submeter teste
+        </button>
       </div>
     </div>
   </form>
@@ -138,11 +140,16 @@
       <div class="modal-body">
         O teste será enviado para classificação e já não poderá voltar atrás.
         Antes de submeter, verifique se respondeu a todas as questões.
-        Desactive as perguntas que não pretende classificar para evitar eventuais penalizações.
+        Desactive as perguntas que não pretende classificar para evitar
+        eventuais penalizações.
       </div>
       <div class="modal-footer">
-        <button type="button" class="btn btn-danger btn-lg" data-dismiss="modal">Oops, NÃO!!!</button>
-        <button form="test" type="submit" class="btn btn-success btn-lg">Sim, submeter...</button>
+        <button type="button" class="btn btn-danger btn-lg" data-dismiss="modal">
+          Oops, NÃO!!!
+        </button>
+        <button form="test" type="submit" class="btn btn-success btn-lg">
+          Sim, submeter...
+        </button>
       </div>
     </div>
   </div>
@@ -7,9 +7,8 @@ from datetime import datetime
 import json
 import logging
 from math import nan
-from os import path
  
-# Logger configuration
+
 logger = logging.getLogger(__name__)
  
  
@@ -30,17 +29,17 @@ class Test(dict):
     '''
  
     # ------------------------------------------------------------------------
-    def __init__(self, d):
+    def __init__(self, d: dict):
         super().__init__(d)
         self['grade'] = nan
         self['comment'] = ''
  
     # ------------------------------------------------------------------------
-    def start(self, student: dict) -> None:
+    def start(self, uid: str) -> None:
         '''
-        Write student id in the test and register start time
+        Register student id and start time in the test
         '''
-        self['student'] = student
+        self['student'] = uid
         self['start_time'] = datetime.now()
         self['finish_time'] = None
         self['state'] = 'ACTIVE'
@@ -57,14 +56,14 @@ class Test(dict):
         self['questions'][ref].set_answer(ans)
  
     # ------------------------------------------------------------------------
-    def submit(self, answers_dict) -> None:
+    def submit(self, answers: dict) -> None:
         '''
         Given a dictionary ans={'ref': 'some answer'} updates the answers of
         multiple questions in the test.
         Only affects the questions referred in the dictionary.
         '''
         self['finish_time'] = datetime.now()
-        for ref, ans in answers_dict.items():
+        for ref, ans in answers.items():
             self['questions'][ref].set_answer(ans)
         self['state'] = 'SUBMITTED'
  
@@ -104,15 +103,11 @@ class Test(dict):
         self['grade'] = 0.0
  
     # ------------------------------------------------------------------------
-    def save_json(self, pathfile) -> None:
+    def save_json(self, filename: str) -> None:
         '''save test in JSON format'''
-        with open(pathfile, 'w') as file:
+        with open(filename, 'w', encoding='utf-8') as file:
             json.dump(self, file, indent=2, default=str)  # str for datetime
  
     # ------------------------------------------------------------------------
     def __str__(self) -> str:
         return '\n'.join([f'{k}: {v}' for k,v in self.items()])
-        # return ('Test:\n'
-        #         f'  student:  {self.get("student", "--")}\n'
-        #         f'  start_time:  {self.get("start_time", "--")}\n'
-        #         f'  questions:  {", ".join(q["ref"] for q in self["questions"])}\n')
@@ -3,20 +3,65 @@ TestFactory - generates tests for students
 '''
  
 # python standard library
+import asyncio
 from os import path
 import random
 import logging
-import re
-from typing import Any, Dict
+
+# other libraries
+import schema
  
 # this project
-from perguntations.questions import QFactory, QuestionException
-from perguntations.test import Test
-from perguntations.tools import load_yaml
+from .questions import QFactory, QuestionException, QDict
+from .test import Test
+from .tools import load_yaml
  
 # Logger configuration
 logger = logging.getLogger(__name__)
  
+# --- test validation --------------------------------------------------------
+def check_answers_directory(ans: str) -> bool:
+    '''Checks is answers_dir exists and is writable'''
+    testfile = path.join(path.expanduser(ans), 'REMOVE-ME')
+    try:
+        with open(testfile, 'w', encoding='utf-8') as file:
+            file.write('You can safely remove this file.')
+    except OSError:
+        return False
+    return True
+
+def check_import_files(files: list) -> bool:
+    '''Checks if the question files exist'''
+    if not files:
+        return False
+    for file in files:
+        if not path.isfile(file):
+            return False
+    return True
+
+def normalize_question_list(questions: list) -> None:
+    '''convert question ref from string to list of string'''
+    for question in questions:
+        if isinstance(question['ref'], str):
+            question['ref'] = [question['ref']]
+
+test_schema = schema.Schema({
+    'ref': schema.Regex('^[a-zA-Z0-9_-]+$'),
+    'database': schema.And(str, path.isfile),
+    'answers_dir': schema.And(str, check_answers_directory),
+    'title': str,
+    schema.Optional('duration'): int,
+    schema.Optional('autosubmit'): bool,
+    schema.Optional('autocorrect'): bool,
+    schema.Optional('show_points'): bool,
+    schema.Optional('scale'): schema.And([schema.Use(float)],
+                                         lambda s: len(s) == 2),
+    'files': schema.And([str], check_import_files),
+    'questions': [{
+        'ref': schema.Or(str, [str]),
+        schema.Optional('points'): float
+        }]
+    }, ignore_extra_keys=True)
  
 # ============================================================================
 class TestFactoryException(Exception):
@@ -32,34 +77,30 @@ class TestFactory(dict):
     '''
  
     # ------------------------------------------------------------------------
-    def __init__(self, conf: Dict[str, Any]) -> None:
+    def __init__(self, conf) -> None:
         '''
         Loads configuration from yaml file, then overrides some configurations
         using the conf argument.
         Base questions are added to a pool of questions factories.
         '''
  
+        test_schema.validate(conf)
+
         # --- set test defaults and then use given configuration
         super().__init__({  # defaults
-            'title': '',
             'show_points': True,
             'scale': None,
             'duration': 0,  # 0=infinite
             'autosubmit': False,
             'autocorrect': True,
-            'debug': False,
-            'show_ref': False,
             })
         self.update(conf)
+        normalize_question_list(self['questions'])
  
         # --- for review, we are done. no factories needed
-        if self['review']:
-            logger.info('Review mode. No questions loaded. No factories.')
-            return
-
-        # --- perform sanity checks and normalize the test questions
-        self.sanity_checks()
-        logger.info('Sanity checks PASSED.')
+        # if self['review']:  FIXME
+        #     logger.info('Review mode. No questions loaded. No factories.')
+        #     return
  
         # --- find refs of all questions used in the test
         qrefs = {r for qq in self['questions'] for r in qq['ref']}
@@ -70,7 +111,7 @@ class TestFactory(dict):
         self['question_factory'] = {}
  
         for file in self["files"]:
-            fullpath = path.normpath(path.join(self["questions_dir"], file))
+            fullpath = path.normpath(file)
  
             logger.info('Loading "%s"...', fullpath)
             questions = load_yaml(fullpath)   # , default=[])
@@ -81,32 +122,25 @@ class TestFactory(dict):
                     msg = f'Question {i} in {file} is not a dictionary'
                     raise TestFactoryException(msg)
  
-                # check if ref is missing, then set to '/path/file.yaml:3'
+                # check if ref is missing, then set to '//file.yaml:3'
                 if 'ref' not in question:
                     question['ref'] = f'{file}:{i:04}'
                     logger.warning('Missing ref set to "%s"', question["ref"])
  
                 # check for duplicate refs
-                if question['ref'] in self['question_factory']:
-                    other = self['question_factory'][question['ref']]
+                qref = question['ref']
+                if qref in self['question_factory']:
+                    other = self['question_factory'][qref]
                     otherfile = path.join(other.question['path'],
                                           other.question['filename'])
-                    msg = (f'Duplicate reference "{question["ref"]}" in files '
-                           f'"{otherfile}" and "{fullpath}".')
+                    msg = f'Duplicate "{qref}" in {otherfile} and {fullpath}'
                     raise TestFactoryException(msg)
  
                 # make factory only for the questions used in the test
-                if question['ref'] in qrefs:
+                if qref in qrefs:
                     question.update(zip(('path', 'filename', 'index'),
                                         path.split(fullpath) + (i,)))
-                    if question['type'] == 'code' and 'server' not in question:
-                        try:
-                            question['server'] = self['jobe_server']
-                        except KeyError as exc:
-                            msg = f'Missing JOBE server in "{question["ref"]}"'
-                            raise TestFactoryException(msg) from exc
-
-                    self['question_factory'][question['ref']] = QFactory(question)
+                    self['question_factory'][qref] = QFactory(QDict(question))
  
         qmissing = qrefs.difference(set(self['question_factory'].keys()))
         if qmissing:
@@ -118,157 +152,124 @@ class TestFactory(dict):
  
  
     # ------------------------------------------------------------------------
-    def check_test_ref(self) -> None:
-        '''Test must have a `ref`'''
-        if 'ref' not in self:
-            raise TestFactoryException('Missing "ref" in configuration!')
-        if not re.match(r'^[a-zA-Z0-9_-]+$', self['ref']):
-            raise TestFactoryException('Test "ref" can only contain the '
-                                       'characters a-zA-Z0-9_-')
-
-    def check_missing_database(self) -> None:
-        '''Test must have a database'''
-        if 'database' not in self:
-            raise TestFactoryException('Missing "database" in configuration')
-        if not path.isfile(path.expanduser(self['database'])):
-            msg = f'Database "{self["database"]}" not found!'
-            raise TestFactoryException(msg)
-
-    def check_missing_answers_directory(self) -> None:
-        '''Test must have a answers directory'''
-        if 'answers_dir' not in self:
-            msg = 'Missing "answers_dir" in configuration'
-            raise TestFactoryException(msg)
-
-    def check_answers_directory_writable(self) -> None:
-        '''Answers directory must be writable'''
-        testfile = path.join(path.expanduser(self['answers_dir']), 'REMOVE-ME')
-        try:
-            with open(testfile, 'w') as file:
-                file.write('You can safely remove this file.')
-        except OSError as exc:
-            msg = f'Cannot write answers to directory "{self["answers_dir"]}"'
-            raise TestFactoryException(msg) from exc
-
-    def check_questions_directory(self) -> None:
-        '''Check if questions directory is missing or not accessible.'''
-        if 'questions_dir' not in self:
-            logger.warning('Missing "questions_dir". Using "%s"',
-                           path.abspath(path.curdir))
-            self['questions_dir'] = path.curdir
-        elif not path.isdir(path.expanduser(self['questions_dir'])):
-            raise TestFactoryException(f'Can\'t find questions directory '
-                                       f'"{self["questions_dir"]}"')
-
-    def check_import_files(self) -> None:
-        '''Check if there are files to import (with questions)'''
-        if 'files' not in self:
-            msg = ('Missing "files" in configuration with the list of '
-                   'question files to import!')
-            raise TestFactoryException(msg)
-
-        if isinstance(self['files'], str):
-            self['files'] = [self['files']]
-
-    def check_question_list(self) -> None:
-        '''normalize question list'''
-        if 'questions' not in self:
-            raise TestFactoryException('Missing "questions" in configuration')
-
-        for i, question in enumerate(self['questions']):
-            # normalize question to a dict and ref to a list of references
-            if isinstance(question, str):  # e.g.,    - some_ref
-                question = {'ref': [question]}    # becomes  - ref: [some_ref]
-            elif isinstance(question, dict) and isinstance(question['ref'], str):
-                question['ref'] = [question['ref']]
-            elif isinstance(question, list):
-                question = {'ref': [str(a) for a in question]}
-
-            self['questions'][i] = question
-
-    def check_missing_title(self) -> None:
-        '''Warns if title is missing'''
-        if not self['title']:
-            logger.warning('Title is undefined!')
-
-    def check_grade_scaling(self) -> None:
-        '''Just informs the scale limits'''
-        if 'scale_points' in self:
-            msg = ('*** DEPRECATION WARNING: *** scale_points, scale_min, '
-                   'scale_max were replaced by "scale: [min, max]".')
-            logger.warning(msg)
-            self['scale'] = [self['scale_min'], self['scale_max']]
+    # def check_test_ref(self) -> None:
+    #     '''Test must have a `ref`'''
+    #     if 'ref' not in self:
+    #         raise TestFactoryException('Missing "ref" in configuration!')
+    #     if not re.match(r'^[a-zA-Z0-9_-]+$', self['ref']):
+    #         raise TestFactoryException('Test "ref" can only contain the '
+    #                                    'characters a-zA-Z0-9_-')
+
+    # def check_missing_database(self) -> None:
+    #     '''Test must have a database'''
+    #     if 'database' not in self:
+    #         raise TestFactoryException('Missing "database" in configuration')
+    #     if not path.isfile(path.expanduser(self['database'])):
+    #         msg = f'Database "{self["database"]}" not found!'
+    #         raise TestFactoryException(msg)
+
+    # def check_missing_answers_directory(self) -> None:
+    #     '''Test must have a answers directory'''
+    #     if 'answers_dir' not in self:
+    #         msg = 'Missing "answers_dir" in configuration'
+    #         raise TestFactoryException(msg)
+
+    # def check_answers_directory_writable(self) -> None:
+    #     '''Answers directory must be writable'''
+    #     testfile = path.join(path.expanduser(self['answers_dir']), 'REMOVE-ME')
+    #     try:
+    #         with open(testfile, 'w', encoding='utf-8') as file:
+    #             file.write('You can safely remove this file.')
+    #     except OSError as exc:
+    #         msg = f'Cannot write answers to directory "{self["answers_dir"]}"'
+    #         raise TestFactoryException(msg) from exc
+
+    # def check_questions_directory(self) -> None:
+    #     '''Check if questions directory is missing or not accessible.'''
+    #     if 'questions_dir' not in self:
+    #         logger.warning('Missing "questions_dir". Using "%s"',
+    #                        path.abspath(path.curdir))
+    #         self['questions_dir'] = path.curdir
+    #     elif not path.isdir(path.expanduser(self['questions_dir'])):
+    #         raise TestFactoryException(f'Can\'t find questions directory '
+    #                                    f'"{self["questions_dir"]}"')
+
+    # def check_import_files(self) -> None:
+    #     '''Check if there are files to import (with questions)'''
+    #     if 'files' not in self:
+    #         msg = ('Missing "files" in configuration with the list of '
+    #                'question files to import!')
+    #         raise TestFactoryException(msg)
+
+        # if isinstance(self['files'], str):
+        #     self['files'] = [self['files']]
+
+    # def check_question_list(self) -> None:
+    #     '''normalize question list'''
+    #     if 'questions' not in self:
+    #         raise TestFactoryException('Missing "questions" in configuration')
+
+    #     for i, question in enumerate(self['questions']):
+    #         # normalize question to a dict and ref to a list of references
+    #         if isinstance(question, str):  # e.g.,    - some_ref
+    #             logger.warning(f'Question "{question}" should be a dictionary')
+    #             question = {'ref': [question]}    # becomes  - ref: [some_ref]
+    #         elif isinstance(question, dict) and isinstance(question['ref'], str):
+    #             question['ref'] = [question['ref']]
+    #         elif isinstance(question, list):
+    #             question = {'ref': [str(a) for a in question]}
+
+    #         self['questions'][i] = question
+
+    # def check_missing_title(self) -> None:
+    #     '''Warns if title is missing'''
+    #     if not self['title']:
+    #         logger.warning('Title is undefined!')
+
+    # def check_grade_scaling(self) -> None:
+    #     '''Just informs the scale limits'''
+    #     if 'scale_points' in self:
+    #         msg = ('*** DEPRECATION WARNING: *** scale_points, scale_min, '
+    #                'scale_max were replaced by "scale: [min, max]".')
+    #         logger.warning(msg)
+    #         self['scale'] = [self['scale_min'], self['scale_max']]
  
  
     # ------------------------------------------------------------------------
-    def sanity_checks(self) -> None:
-        '''
-        Checks for valid keys and sets default values.
-        Also checks if some files and directories exist
-        '''
-        self.check_test_ref()
-        self.check_missing_database()
-        self.check_missing_answers_directory()
-        self.check_answers_directory_writable()
-        self.check_questions_directory()
-        self.check_import_files()
-        self.check_question_list()
-        self.check_missing_title()
-        self.check_grade_scaling()
+    # def sanity_checks(self) -> None:
+    #     '''
+    #     Checks for valid keys and sets default values.
+    #     Also checks if some files and directories exist
+    #     '''
+        # self.check_test_ref()
+        # self.check_missing_database()
+        # self.check_missing_answers_directory()
+        # self.check_answers_directory_writable()
+        # self.check_questions_directory()
+        # self.check_import_files()
+        # self.check_question_list()
+        # self.check_missing_title()
+        # self.check_grade_scaling()
  
     # ------------------------------------------------------------------------
     def check_questions(self) -> None:
         '''
         checks if questions can be correctly generated and corrected
         '''
-        logger.info('Checking if questions can be generated and corrected...')
+        logger.info('Checking questions...')
+        # FIXME get_event_loop will be deprecated in python3.10
+        loop = asyncio.get_event_loop()
         for i, (qref, qfact) in enumerate(self['question_factory'].items()):
             try:
-                question = qfact.generate()
+                question = loop.run_until_complete(qfact.gen_async())
             except Exception as exc:
                 msg = f'Failed to generate "{qref}"'
                 raise TestFactoryException(msg) from exc
             else:
                 logger.info('%4d.  %s:  Ok', i, qref)
-                # logger.info('         generate Ok')
-
-            if question['type'] in ('code', 'textarea'):
-                if 'tests_right' in question:
-                    for tnum, right_answer in enumerate(question['tests_right']):
-                        try:
-                            question.set_answer(right_answer)
-                            question.correct()
-                        except Exception as exc:
-                            msg = f'Failed to correct "{qref}"'
-                            raise TestFactoryException(msg) from exc
-
-                        if question['grade'] == 1.0:
-                            logger.info('         test %i Ok', tnum)
-                        else:
-                            logger.error('         TEST %i IS WRONG!!!', tnum)
-                elif 'tests_wrong' in question:
-                    for tnum, wrong_answer in enumerate(question['tests_wrong']):
-                        try:
-                            question.set_answer(wrong_answer)
-                            question.correct()
-                        except Exception as exc:
-                            msg = f'Failed to correct "{qref}"'
-                            raise TestFactoryException(msg) from exc
-
-                        if question['grade'] < 1.0:
-                            logger.info('         test %i Ok', tnum)
-                        else:
-                            logger.error('         TEST %i IS WRONG!!!', tnum)
-                else:
-                    try:
-                        question.set_answer('')
-                        question.correct()
-                    except Exception as exc:
-                        msg = f'Failed to correct "{qref}"'
-                        raise TestFactoryException(msg) from exc
-                    else:
-                        logger.info('         correct Ok but no tests to run')
  
+            if question['type'] == 'textarea':
+                _runtests_textarea(qref, question)
     # ------------------------------------------------------------------------
     async def generate(self):
         '''
@@ -326,16 +327,51 @@ class TestFactory(dict):
             logger.error('%s errors found!', nerr)
  
         # copy these from the test configuratoin to each test instance
-        inherit = {'ref', 'title', 'database', 'answers_dir',
-                   'questions_dir', 'files',
-                   'duration', 'autosubmit', 'autocorrect',
-                   'scale', 'show_points',
-                   'show_ref', 'debug', }
-        # NOT INCLUDED:  testfile, allow_all, review
+        inherit = ['ref', 'title', 'database', 'answers_dir', 'files', 'scale',
+                   'duration', 'autosubmit', 'autocorrect', 'show_points']
  
         return Test({'questions': questions, **{k:self[k] for k in inherit}})
  
     # ------------------------------------------------------------------------
     def __repr__(self):
         testsettings = '\n'.join(f'  {k:14s}: {v}' for k, v in self.items())
-        return '{\n' + testsettings + '\n}'
+        return 'TestFactory({\n' + testsettings + '\n})'
+
+# ============================================================================
+def _runtests_textarea(qref, question):
+    '''
+    Checks if correction script works and runs tests if available
+    '''
+    try:
+        question.set_answer('')
+        question.correct()
+    except Exception as exc:
+        msg = f'Failed to correct "{qref}"'
+        raise TestFactoryException(msg) from exc
+    logger.info('         correction works')
+
+    for tnum, right_answer in enumerate(question.get('tests_right', {})):
+        try:
+            question.set_answer(right_answer)
+            question.correct()
+        except Exception as exc:
+            msg = f'Failed to correct "{qref}"'
+            raise TestFactoryException(msg) from exc
+
+        if question['grade'] == 1.0:
+            logger.info('         tests_right[%i] Ok', tnum)
+        else:
+            logger.error('         tests_right[%i] FAILED!!!', tnum)
+
+    for tnum, wrong_answer in enumerate(question.get('tests_wrong', {})):
+        try:
+            question.set_answer(wrong_answer)
+            question.correct()
+        except Exception as exc:
+            msg = f'Failed to correct "{qref}"'
+            raise TestFactoryException(msg) from exc
+
+        if question['grade'] < 1.0:
+            logger.info('         tests_wrong[%i] Ok', tnum)
+        else:
+            logger.error('         tests_wrong[%i] FAILED!!!', tnum)
 '''
-This module contains helper functions to:
-- load yaml files and report errors
-- run external programs (sync and async)
+File:           perguntations/tools.py
+Description:    Helper functions to load yaml files and run external programs.
 '''
  
  
@@ -20,28 +19,11 @@ import yaml
 logger = logging.getLogger(__name__)
  
  
-# ---------------------------------------------------------------------------
-def load_yaml(filename: str, default: Any = None) -> Any:
-    '''load data from yaml file'''
-
-    filename = path.expanduser(filename)
-    try:
-        file = open(filename, 'r', encoding='utf-8')
-    except Exception as exc:
-        logger.error(exc)
-        if default is not None:
-            return default
-        raise
-
-    with file:
-        try:
-            return yaml.safe_load(file)
-        except yaml.YAMLError as exc:
-            logger.error(str(exc).replace('\n', ' '))
-            if default is not None:
-                return default
-            raise
-
+# ----------------------------------------------------------------------------
+def load_yaml(filename: str) -> Any:
+    '''load yaml file or raise exception on error'''
+    with open(path.expanduser(filename), 'r', encoding='utf-8') as file:
+        return yaml.safe_load(file)
  
 # ---------------------------------------------------------------------------
 def run_script(script: str,
@@ -53,7 +35,7 @@ def run_script(script: str,
     The script is run in another process but this function blocks waiting
     for its termination.
     '''
-    logger.info('run_script "%s"', script)
+    logger.debug('run_script "%s"', script)
  
     output = None
     script = path.expanduser(script)
@@ -22,14 +22,15 @@ setup(
     url="https://git.xdi.uevora.pt/mjsb/perguntations.git",
     packages=find_packages(),
     include_package_data=True,   # install files from MANIFEST.in
-    python_requires='>=3.7.*',
+    python_requires='>=3.8.*',
     install_requires=[
-        'tornado>=6.0',
-        'mistune',
+        'bcrypt>=3.1',
+        'mistune<2.0',
         'pyyaml>=5.1',
         'pygments',
-        'sqlalchemy',
-        'bcrypt>=3.1'
+        'schema>=0.7.5',
+        'sqlalchemy>=1.4',
+        'tornado>=6.1',
         ],
     entry_points={
         'console_scripts': [
@@ -3,4 +3,3 @@
 git pull
 npm update
 pip install -U .
-
...	...	@@ -11,26 +11,27 @@
11	11
12	12	## Requirements
13	13
14		-The webserver is a python application that requires `>=python3.7` and the package installer for python `pip`. The node package management `npm` is also necessary in order to install the javascript libraries.
	14	+The webserver is a python application that requires `>=python3.8` and the
	15	+package installer for python `pip`. The node package management `npm` is also
	16	+necessary in order to install the javascript libraries.
15	17
16	18	```sh
17	19	sudo apt install python3 python3-pip npm # Ubuntu
18		-sudo pkg install python37 py37-sqlite3 py37-pip npm # FreeBSD
19		-sudo port install python37 py37-pip py37-setuptools npm6 # MacOS (macports)
	20	+sudo pkg install python38 py38-sqlite3 py38-pip npm # FreeBSD
	21	+sudo port install python38 py38-pip py38-setuptools npm6 # MacOS (macports)
20	22	```
21	23
22		-To make the `pip` install packages to a local directory, the file `pip.conf` should be configured as follows:
	24	+To make the `pip` install packages to a local directory, the file `pip.conf`
	25	+should be configured as follows:
23	26
24	27	```ini
25	28	[global]
26	29	user=yes
27		-
28		-[list]
29		-format=columns
30	30	```
31	31
32	32	This file is usually in `~/.config/pip/` in Linux and FreeBSD. In MacOS it's in
33		-`~/Library/Application Support/pip/`. You may have to create it, if it doesn't exist yet.
	33	+`~/Library/Application Support/pip/`. You may have to create it, if it doesn't
	34	+exist yet.
34	35
35	36	---
36	37
...	...	@@ -42,19 +43,21 @@ Download and install:
42	43	git clone https://git.xdi.uevora.pt/mjsb/perguntations.git
43	44	cd perguntations
44	45	npm install
45		-pip3 install .
	46	+pip install .
46	47	```
47	48
48		-The command `npm` installs the javascript libraries and then `pip3` installs the python webserver. This will also install any required dependencies.
	49	+The command `npm` installs the javascript libraries and then `pip` installs the
	50	+python webserver. This will also install any required dependencies.
49	51
50		-Atention! `pip3 install .` must run after `npm install`, otherwise the javascript libraries will not be found during the install.
	52	+Atention! `pip install .` must be run after `npm install`, otherwise
	53	+the javascript libraries will not be found during the install.
51	54
52	55	---
53	56
54	57	## Setup
55	58
56		-The server will run a `https` server and requires valid certificates.
57		-There are two possibilities to generate the certificates:
	59	+The server will run a `https` server and requires valid certificates. There
	60	+are two possibilities to generate the certificates:
58	61
59	62	- public server with static IP address and registered domain name;
60	63	- private server on a local network isolated from the internet.
...	...	@@ -74,38 +77,35 @@ cd ~/.local/share/certs
74	77	openssl req -x509 -newkey rsa:4096 -keyout privkey.pem -out cert.pem -days 365 -nodes
75	78	```
76	79
77		-To have a true certificate, like the ones issued by LetsEncrypt, the host
78		-must have a registered domain name.
79		-See instructions for FreeBSD at the end of this page.
	80	+To have a true certificate, like the ones issued by LetsEncrypt, the host must
	81	+have a registered domain name. See instructions for FreeBSD at the end of this
	82	+page.
80	83
81	84	---
82	85
83	86	## Running a demo
84	87
85		-The directory `demo` in the repository includes a demo test that can be used
86		-as a template for your own tests and questions.
87		-
88		-To run the demonstration test you need to initialize the database using one of
89		-the following methods:
	88	+The directory `demo` in the repository includes a demo test that can be used as
	89	+a template for your own tests and questions. To run the demonstration test you
	90	+need to initialize the database using one of the following methods:
90	91
91	92	```sh
92	93	cd perguntations/demo
93		-
94	94	initdb students.csv # initialize from a CSV file
95	95	initdb --admin # only adds the administrator account
96	96	initdb --add 123 "Asterix Gaules" # add one student
97	97	```
98	98
99	99	This will create or update a `students.db` file that contains a sqlite3
100		-database.
101		-The database stores user passwords and grades, but not the actual tests.
	100	+database. The database stores user passwords and grades, but not the actual
	101	+tests.
102	102
103		-A test is specified in a single `yaml` file.
104		-The demo already includes the `demo.yaml` that you can play with.
	103	+A test is specified in a single `yaml` file. The file `demo.yaml` is a test
	104	+specification that you can play with.
105	105
106		-The complete tests submitted by the students are stored in JSON files in the
107		-directory defined in `demo.yaml` under the option `answers_dir: ans`.
108		-We also have to create this directory manually:
	106	+The answered tests submitted by the students are stored in JSON files in a
	107	+directory defined in `demo.yaml` under the option `answers_dir: ans`. We also
	108	+have to create this directory manually:
109	109
110	110	```sh
111	111	mkdir ans # directory where the tests will be saved
...	...	@@ -114,38 +114,36 @@ mkdir ans # directory where the tests will be saved
114	114	Start the server and run the `demo.yaml` test:
115	115
116	116	```sh
117		-perguntations demo.yaml # run demo test
	117	+perguntations demo.yaml # run demo test
118	118	```
119	119
120	120	Several options are available, run `perguntations --help` for a list.
121	121
122		-The server listens on port 8443 of all IPs of all network interfaces.
123		-Open the browser at `http://127.0.0.1:8443/` and login as user number `0`
	122	+The server listens on port 8443 of all IPs of all network interfaces. Open the
	123	+browser at `http://127.0.0.1:8443/` and login as user number `0`
124	124	(administrator) and choose any password you want. The password is defined on
125	125	first login.
	126	+
126	127	After logging in, you will be redirected to the administration page that shows
127	128	all the students and their current state.
128	129
129	130	1. Authorize students by clicking the checkboxes.
130	131	2. Open a different browser (or exit administrator) at `http://127.0.0.1:8443/`
131		-and login as one of the authorized students.
132		-Answer the questions and submit. You should get a grade at the end.
	132	+ and login as one of the authorized students. Answer the questions and
	133	+ submit. You should get a grade at the end.
133	134
134		-The server can be stoped from the terminal with `^C`.
	135	+The server can be stoped from the terminal with `^C` (Control+C).
135	136
136	137	---
137	138
138	139	## Running on lower ports
139	140
140	141	Ports 80 and 443 are reserved for the root user and and this software **should
141		-NOT be run as root**.
142		-Instead, tcp traffic can be forwarded from port 443 to 8443 where the server is
143		-listening.
144		-The details depend on the operating system/firewall.
145		-
146		-### debian
	142	+NOT be run as root**. Instead, tcp traffic can be forwarded from port 443 to
	143	+8443 where the server is listening. The details depend on the operating
	144	+system/firewall.
147	145
148		-FIXME: Untested
	146	+### debian (FIXME: Untested)
149	147
150	148	```.sh
151	149	iptables -t nat -I PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-ports 8443
...	...	@@ -166,25 +164,29 @@ Explanation:
166	164
167	165	Edit `/etc/pf.conf`:
168	166
169		- ext_if="wlan1"
170		- rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
171		- rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
	167	+```conf
	168	+ext_if="wlan1"
	169	+rdr on $ext_if proto tcp from any to any port 80 -> 127.0.0.1 port 8080
	170	+rdr on $ext_if proto tcp from any to any port 443 -> 127.0.0.1 port 8443
	171	+```
172	172
173		-The `wlan1` should be the name of the network interface.
174		-Use `ext_if="vtnet0"` for guest additions under virtual box.
	173	+The `wlan1` should be the name of the network interface. Use `ext_if="vtnet0"`
	174	+for guest additions under virtual box.
175	175
176	176	Start firewall with `sudo service pf start`.
177	177
178	178	Optionally, to activate pf on boot, edit `rc.conf`:
179	179
180		- pf_enable="YES"
181		- pf_flags=""
182		- pf_rules="/etc/pf.conf"
	180	+```conf
	181	+pf_enable="YES"
	182	+pf_flags=""
	183	+pf_rules="/etc/pf.conf"
183	184
184		- # optional logging:
185		- pflog_enable="YES"
186		- pflog_flags=""
187		- pflog_logfile="/var/log/pflog"
	185	+# optional logging:
	186	+pflog_enable="YES"
	187	+pflog_flags=""
	188	+pflog_logfile="/var/log/pflog"
	189	+```
188	190
189	191	## Generating certificates with LetsEncript (FreeBSD)
190	192
...	...	@@ -192,7 +194,7 @@ Generating certificates for a public server (FreeBSD) requires a registered
192	194	domain with fixed IP.
193	195
194	196	```sh
195		-sudo pkg install py27-certbot # FreeBSD
	197	+sudo pkg install py38-certbot # FreeBSD 13
196	198	sudo service pf stop # disable pf firewall (FreeBSD)
197	199	sudo certbot certonly --standalone -d www.example.com
198	200	sudo service pf start # enable pf firewall
...	...	@@ -208,47 +210,52 @@ chmod 400 cert.pem privkey.pem
208	210	Certificate renewals can be done as follows:
209	211
210	212	```sh
211		-sudo service pf stop # shutdown firewall
	213	+sudo service pf stop # shutdown firewall
212	214	sudo certbot renew
213		-sudo service pf start # start firewall
	215	+sudo service pf start # start firewall
214	216	```
215	217
216		-Again, copy certificate files `privkey.pem` and `cert.pem` to `~/.local/share/certs`.
	218	+Again, copy certificate files `privkey.pem` and `cert.pem` to
	219	+`~/.local/share/certs`.
217	220
218	221	---
219	222
220	223	## Upgrading
221	224
222		-From time to time there can be updates to perguntations, python packages and javascript libraries.
223		-
224		-Python packages can be upgraded independently of the rest using pip:
	225	+From time to time there can be updates to perguntations, python packages and
	226	+javascript libraries. Python packages can be upgraded independently of the
	227	+rest using pip:
225	228
226	229	```sh
227		-pip list --outdated # lists upgradable packages
	230	+pip list --outdated --user # lists upgradable packages
228	231	pip install -U something # upgrade something
229	232	```
230	233
	234	+Attention: don't upgrade blindly. Some upgrades can be incompatible with
	235	+perguntations and break things! All upgrades require testing of all the
	236	+funcionalities.
	237	+
231	238	To upgrade perguntations and javascript libraries do:
232	239
233	240	```sh
234	241	cd perguntations
235	242	git pull # get latest version of perguntations
236	243	npm update # get latest versions of javascript libraries
237		-pip3 install -U . # upgrade perguntations
	244	+pip install -U . # upgrade perguntations
238	245	```
239	246
240	247	## Troubleshooting
241	248
242	249	- The server tries to run `python3` so this command must be accessible from
243		-user accounts. Currently, the minimum supported python version is 3.7.
244		-
	250	+ user accounts. Currently, the minimum supported python version is 3.8.
245	251	- If you are getting any `UnicodeEncodeError` type of errors that's because the
246		-terminal is not supporting UTF-8.
247		-This error may occur when a unicode character is printed to the screen by the
248		-server or, when running question generator or correction scripts, a message is
249		-piped between the server and the scripts that includes unicode characters.
250		-Try running `locale` on the terminal and see if there are any error messages.
251		-Solutions:
	252	+ terminal is not supporting UTF-8. This error may occur when a unicode
	253	+ character is printed to the screen by the server or, when running question
	254	+ generator or correction scripts, a message is piped between the server and
	255	+ the scripts that includes unicode characters. Try running `locale` on the
	256	+ terminal and see if there are any error messages.
	257	+ Solutions:
	258	+
252	259	- debian: `sudo dpkg-reconfigure locales` and select your UTF-8 locales.
253	260	- FreeBSD: edit `~/.login_conf` to use UTF-8, for example:
254	261
...	...
...	...	@@ -14,9 +14,6 @@ database: students.db
14	14	# Directory where the submitted and corrected test are stored for later review.
15	15	answers_dir: ans
16	16
17		-# Server used to compile & execute code
18		-jobe_server: 192.168.1.85
19		-
20	17	# --- optional settings: -----------------------------------------------------
21	18
22	19	# Title of this test, e.g. course name, year or test number
...	...	@@ -41,43 +38,46 @@ autocorrect: true
41	38	# (default: true)
42	39	show_points: true
43	40
44		-# scale final grade to an interval, e.g. [0, 20], keeping the relative weight
45		-# of the points declared in the questions below.
	41	+# Scale the points of the questions so that the final grade is in the given
	42	+# interval.
46	43	# (default: no scaling, just use question points)
47		-scale: [0, 5]
	44	+scale: [0, 20]
48	45
49	46
50	47	# ----------------------------------------------------------------------------
51		-# Base path applied to the questions files and all the scripts
52		-# including question generators and correctors.
53		-# Either absolute path or relative to current directory can be used.
54		-questions_dir: .
55		-
56		-# (optional) List of files containing questions in yaml format.
57		-# Selected questions will be obtained from these files.
58		-# If undefined, all yaml files in questions_dir are loaded (not recommended).
	48	+# Files to import. Each file contains a list of questions in yaml format.
59	49	files:
60	50	- questions/questions-tutorial.yaml
61	51
62	52	# This is the list of questions that will make up the test.
63	53	# The order is preserved.
64		-# There are several ways to define each question (explained below).
	54	+# Each question is a dictionary with a question `ref` or a list of `ref`.
	55	+# If a list is given, one question will be choosen randomly to each student.
	56	+# The `points` for each question is optional and is 1.0 by default for normal
	57	+# questions. Informative type of "questions" will have 0.0 points.
	58	+# Points are automatically scaled if `scale` key is defined.
65	59	questions:
66	60	- ref: tut-test
67		- - tut-questions
	61	+ - ref: tut-questions
	62	+
	63	+ # these will have 1.0 points
	64	+ - ref: tut-radio
	65	+ - ref: tut-checkbox
	66	+ - ref: tut-text
	67	+ - ref: tut-text-regex
	68	+ - ref: tut-numeric-interval
68	69
69		- - tut-radio
70		- - tut-checkbox
71		- - tut-text
72		- - tut-text-regex
73		- - tut-numeric-interval
	70	+ # this question will have 2.0 points
74	71	- ref: tut-textarea
75	72	points: 2.0
76	73
77		- - tut-information
78		- - tut-success
79		- - tut-warning
80		- - [tut-alert1, tut-alert2]
81		- - tut-generator
82		- - tut-yamllint
83		- # - tut-code
	74	+ # these will have 0.0 points:
	75	+ - ref: tut-information
	76	+ - ref: tut-success
	77	+ - ref: tut-warning
	78	+
	79	+ # choose one from the list:
	80	+ - ref: [tut-alert1, tut-alert2]
	81	+
	82	+ - ref: tut-generator
	83	+ - ref: tut-yamllint
...	...
...	...	@@ -0,0 +1,77 @@
	1	+#!/usr/bin/env python3
	2	+
	3	+'''
	4	+Example of a question generator.
	5	+Arguments are read from stdin.
	6	+'''
	7	+
	8	+from random import randint
	9	+import sys
	10	+
	11	+# read two arguments from the field `args` specified in the question yaml file
	12	+a, b = (int(n) for n in sys.argv[1:])
	13	+
	14	+x = randint(a, b)
	15	+y = randint(a, b)
	16	+r = x + y
	17	+
	18	+print(f"""---
	19	+type: text
	20	+title: Geradores de perguntas
	21	+text: \|
	22	+
	23	+ As perguntas podem ser estáticas (como as que vimos até aqui), ou serem
	24	+ geradas dinâmicamente por um programa externo. Para gerar uma pergunta, o
	25	+ programa deve escrever texto no `stdout` em formato `yaml` tal como os
	26	+ exemplos das perguntas estáticas dos tipos apresentados anteriormente. Pode
	27	+ também receber argumentos de linha de comando para parametrizar a pergunta.
	28	+ Aqui está um exemplo de uma pergunta gerada por um script python:
	29	+
	30	+ ```python
	31	+ #!/usr/bin/env python3
	32	+
	33	+ from random import randint
	34	+ import sys
	35	+
	36	+ a, b = (int(n) for n in sys.argv[1:]) # argumentos da linha de comando
	37	+
	38	+ x = randint(a, b) # número inteiro no intervalo a..b
	39	+ y = randint(a, b) # número inteiro no intervalo a..b
	40	+ r = x + y # calcula resultado correcto
	41	+
	42	+ print(f'''---
	43	+ type: text
	44	+ title: Contas de somar
	45	+ text: \|
	46	+ Calcule o resultado de ${{x}} + {{y}}$.
	47	+ correct: '{{r}}'
	48	+ solution: \|
	49	+ A solução é {{r}}.''')
	50	+ ```
	51	+
	52	+ Este script deve ter permissões para poder ser executado no terminal.
	53	+ Podemos testar o programa no terminal `./gen-somar.py 1 100` e verificar que
	54	+ o output é uma pergunta válida em formato `yaml`. Agora é necessário indicar
	55	+ que este script deve ser usado para gerar uma pergunta.
	56	+
	57	+ Uma pergunta gerada por um programa externo é declarada com
	58	+
	59	+ ```yaml
	60	+ - type: generator
	61	+ ref: gen-somar
	62	+ script: gen-somar.py
	63	+ # argumentos opcionais
	64	+ args: [1, 100]
	65	+ ```
	66	+
	67	+ O programa pode receber uma lista de argumentos de linha de comando
	68	+ declarados em `args`.
	69	+
	70	+ ---
	71	+
	72	+ Calcule o resultado de ${x} + {y}$.
	73	+
	74	+ Os números foram gerados aleatoriamente no intervalo de {a} a {b}.
	75	+correct: '{r}'
	76	+solution: \|
	77	+ A solução é {r}.""")
...	...
...	...	@@ -1,77 +0,0 @@
1		-#!/usr/bin/env python3
2		-
3		-'''
4		-Example of a question generator.
5		-Arguments are read from stdin.
6		-'''
7		-
8		-from random import randint
9		-import sys
10		-
11		-# read two arguments from the field `args` specified in the question yaml file
12		-a, b = (int(n) for n in sys.argv[1:])
13		-
14		-x = randint(a, b)
15		-y = randint(a, b)
16		-r = x + y
17		-
18		-print(f"""---
19		-type: text
20		-title: Geradores de perguntas
21		-text: \|
22		-
23		- As perguntas podem ser estáticas (como as que vimos até aqui), ou serem
24		- geradas dinâmicamente por um programa externo. Para gerar uma pergunta, o
25		- programa deve escrever texto no `stdout` em formato `yaml` tal como os
26		- exemplos das perguntas estáticas dos tipos apresentados anteriormente. Pode
27		- também receber argumentos de linha de comando para parametrizar a pergunta.
28		- Aqui está um exemplo de uma pergunta gerada por um script python:
29		-
30		- ```python
31		- #!/usr/bin/env python3
32		-
33		- from random import randint
34		- import sys
35		-
36		- a, b = (int(n) for n in sys.argv[1:]) # argumentos da linha de comando
37		-
38		- x = randint(a, b) # número inteiro no intervalo a..b
39		- y = randint(a, b) # número inteiro no intervalo a..b
40		- r = x + y # calcula resultado correcto
41		-
42		- print(f'''---
43		- type: text
44		- title: Contas de somar
45		- text: \|
46		- Calcule o resultado de ${{x}} + {{y}}$.
47		- correct: '{{r}}'
48		- solution: \|
49		- A solução é {{r}}.''')
50		- ```
51		-
52		- Este script deve ter permissões para poder ser executado no terminal.
53		- Podemos testar o programa no terminal `./gen-somar.py 1 100` e verificar que
54		- o output é uma pergunta válida em formato `yaml`. Agora é necessário indicar
55		- que este script deve ser usado para gerar uma pergunta.
56		-
57		- Uma pergunta gerada por um programa externo é declarada com
58		-
59		- ```yaml
60		- - type: generator
61		- ref: gen-somar
62		- script: gen-somar.py
63		- # argumentos opcionais
64		- args: [1, 100]
65		- ```
66		-
67		- O programa pode receber uma lista de argumentos de linha de comando
68		- declarados em `args`.
69		-
70		- ---
71		-
72		- Calcule o resultado de ${x} + {y}$.
73		-
74		- Os números foram gerados aleatoriamente no intervalo de {a} a {b}.
75		-correct: '{r}'
76		-solution: \|
77		- A solução é {r}.""")
...	...	@@ -20,24 +20,20 @@
20	20	database: students.db # base de dados previamente criada com initdb
21	21	answers_dir: ans # directório onde ficam os testes dos alunos
22	22
23		- # opcional
	23	+ # opcionais
24	24	duration: 60 # duração da prova em minutos (default: inf)
25	25	autosubmit: true # submissão automática (default: false)
26	26	show_points: true # mostra cotação das perguntas (default: true)
27		- scale: [0, 20] # limites inferior e superior da escala (default: [0,20])
28		- scale_points: true # normaliza cotações para a escala definida
29		- jobe_server: moodle-jobe.uevora.pt # server used to compile & execute code
30		- debug: false # mostra informação de debug no browser
	27	+ scale: [0, 20] # normaliza cotações para o intervalo indicado.
	28	+ # não normaliza por defeito (default: None)
31	29
32	30	# --------------------------------------------------------------------------
33		- questions_dir: ~/topics # raíz da árvore de directórios das perguntas
34		-
35	31	# Ficheiros de perguntas a importar (relativamente a `questions_dir`)
36	32	files:
37	33	- tabelas.yaml
38		- - topic_A/questions.yaml
39		- - topic_B/part_1/questions.yaml
40		- - topic_B/part_2/questions.yaml
	34	+ - topic1/questions.yaml
	35	+ - topic2/part1/questions.yaml
	36	+ - topic2/part2/questions.yaml
41	37
42	38	# --------------------------------------------------------------------------
43	39	# Especificação das perguntas do teste e respectivas cotações.
...	...	@@ -50,13 +46,10 @@
50	46	- ref: pergunta2
51	47	points: 2.0
52	48
53		- # a cotação é 1.0 por defeito
	49	+ # por defeinto, a cotação da pergunta é 1.0 valor
54	50	- ref: pergunta3
55	51
56		- # uma string (não dict), é interpretada como referência
57		- - tabela-auxiliar
58		-
59		- # escolhe aleatoriamente uma das variantes
	52	+ # escolhe aleatoriamente uma das variantes da pergunta
60	53	- ref: [pergunta3a, pergunta3b]
61	54	points: 0.5
62	55
...	...	@@ -96,8 +89,7 @@
96	89	text: \|
97	90	Quando o texto da pergunta tem várias linhas, dá jeito usar o símbolo
98	91	`\|` de pipe, para indicar que tudo o que estiver indentado faz parte do
99		- texto.
100		- É o caso desta pergunta.
	92	+ texto. É o caso desta pergunta.
101	93
102	94	O texto das perguntas é escrito em `markdown` e suporta fórmulas em
103	95	LaTeX.
...	...	@@ -105,8 +97,8 @@
105	97	#---------------------------------------------------------------------------
106	98	```
107	99
108		- As chaves são usadas para construir o teste e não se podem repetir, mesmo em
109		- ficheiros diferentes.
	100	+ As chaves são usadas para construir o teste e não se podem repetir, mesmo
	101	+ em ficheiros diferentes.
110	102	De seguida mostram-se exemplos dos vários tipos de perguntas.
111	103
112	104	# ----------------------------------------------------------------------------
...	...	@@ -431,6 +423,10 @@
431	423	pode estar previamente preenchida como neste caso (use `answer: texto`).
432	424	correct: correct/correct-question.py
433	425	timeout: 5
	426	+ tests_right:
	427	+ - 'red green blue'
	428	+ # tests_wrong:
	429	+ # - 'blue gray yellow'
434	430
435	431	# ---------------------------------------------------------------------------
436	432	- type: information
...	...	@@ -504,6 +500,7 @@
504	500	return 0; // comentario
505	501	}
506	502	```
	503	+
507	504	```
508	505
509	506	# ---------------------------------------------------------------------------
...	...	@@ -586,7 +583,7 @@
586	583	# ----------------------------------------------------------------------------
587	584	- type: generator
588	585	ref: tut-generator
589		- script: generators/generate-question.py
	586	+ script: generate-question.py
590	587	args: [1, 100]
591	588
592	589	# ----------------------------------------------------------------------------
...	...
1	1	[mypy]
2		-python_version = 3.7
3		-# warn_return_any = True
4		-# warn_unused_configs = True
5		-
6		-[mypy-sqlalchemy.*]
	2	+python_version = 3.9
7	3	ignore_missing_imports = True
8	4
9		-[mypy-pygments.*]
10		-ignore_missing_imports = True
11	5
12		-[mypy-bcrypt.*]
13		-ignore_missing_imports = True
	6	+; [mypy-setuptools.*]
	7	+; ignore_missing_imports = True
14	8
15		-[mypy-mistune.*]
16		-ignore_missing_imports = True
	9	+; [mypy-sqlalchemy.*]
	10	+; ignore_missing_imports = True
	11	+
	12	+; [mypy-pygments.*]
	13	+; ignore_missing_imports = True
	14	+
	15	+; [mypy-mistune.*]
	16	+; ignore_missing_imports = True
...	...
...	...	@@ -2,12 +2,12 @@
2	2	"description": "Javascript libraries required to run the server",
3	3	"email": "mjsb@uevora.pt",
4	4	"dependencies": {
5		- "@fortawesome/fontawesome-free": "^5.15.1",
6		- "bootstrap": "^4.5",
7		- "codemirror": "^5.58.1",
	5	+ "@fortawesome/fontawesome-free": "^5.15.3",
	6	+ "bootstrap": "^4.6.0",
	7	+ "codemirror": "^5.61.1",
8	8	"datatables": "^1.10",
9		- "jquery": "^3.5.1",
10		- "mathjax": "^3.1.2",
11		- "underscore": "^1.11.0"
	9	+ "jquery": "^3.6.0",
	10	+ "mathjax": "^3.1.4",
	11	+ "underscore": "^1.13.1"
12	12	}
13	13	}
...	...
1		-# Copyright (C) 2020 Miguel Barão
	1	+# Copyright (C) 2022 Miguel Barão
2	2	#
3	3	# THE MIT License
4	4	#
...	...	@@ -32,10 +32,10 @@ proof of submission and for review.
32	32	'''
33	33
34	34	APP_NAME = 'perguntations'
35		-APP_VERSION = '2021.02.dev1'
36		-APP_DESCRIPTION = __doc__
	35	+APP_VERSION = '2022.01.dev1'
	36	+APP_DESCRIPTION = str(__doc__)
37	37
38	38	__author__ = 'Miguel Barão'
39		-__copyright__ = 'Copyright 2020, Miguel Barão'
	39	+__copyright__ = 'Copyright 2022, Miguel Barão'
40	40	__license__ = 'MIT license'
41	41	__version__ = APP_VERSION
...	...
...	...	@@ -13,17 +13,18 @@ from concurrent.futures import ThreadPoolExecutor
13	13
14	14	# installed packages
15	15	import bcrypt
16		-import sqlalchemy as sa
17		-import sqlalchemy.orm
	16	+from sqlalchemy import create_engine, select
	17	+from sqlalchemy.orm import Session
18	18	from sqlalchemy.exc import IntegrityError
19	19
20	20	# this project
21		-from perguntations.models import Base, Student
	21	+from .models import Base, Student
22	22
23	23
24	24	# ============================================================================
25	25	def parse_commandline_arguments():
26	26	'''Parse command line options'''
	27	+
27	28	parser = argparse.ArgumentParser(
28	29	formatter_class=argparse.ArgumentDefaultsHelpFormatter,
29	30	description='Insert new users into a database. Users can be imported '
...	...	@@ -104,7 +105,7 @@ def get_students_from_csv(filename):
104	105
105	106
106	107	# ============================================================================
107		-def hashpw(student, password=None):
	108	+def hashpw(student, password=None) -> None:
108	109	'''replace password by hash for a single student'''
109	110	print('.', end='', flush=True)
110	111	if password is None:
...	...	@@ -115,12 +116,13 @@ def hashpw(student, password=None):
115	116
116	117
117	118	# ============================================================================
118		-def insert_students_into_db(session, students):
	119	+def insert_students_into_db(session, students) -> None:
119	120	'''insert list of students into the database'''
120	121	try:
121	122	session.add_all([Student(id=s['uid'], name=s['name'], password=s['pw'])
122	123	for s in students])
123	124	session.commit()
	125	+
124	126	except IntegrityError:
125	127	print('!!! Integrity error. Users already in database. Aborted !!!\n')
126	128	session.rollback()
...	...	@@ -129,11 +131,12 @@ def insert_students_into_db(session, students):
129	131	# ============================================================================
130	132	def show_students_in_database(session, verbose=False):
131	133	'''get students from database'''
132		- users = session.query(Student).all()
	134	+ users = session.execute(select(Student)).scalars().all()
	135	+ # users = session.query(Student).all()
	136	+ total = len(users)
133	137
134		- total_users = len(users)
135	138	print('Registered users:')
136		- if total_users == 0:
	139	+ if total == 0:
137	140	print(' -- none --')
138	141	else:
139	142	users.sort(key=lambda u: f'{u.id:>12}') # sort by number
...	...	@@ -142,13 +145,13 @@ def show_students_in_database(session, verbose=False):
142	145	print(f'{user.id:>12} {user.name}')
143	146	else:
144	147	print(f'{users[0].id:>12} {users[0].name}')
145		- if total_users > 1:
	148	+ if total > 1:
146	149	print(f'{users[1].id:>12} {users[1].name}')
147		- if total_users > 3:
	150	+ if total > 3:
148	151	print(' \| \|')
149		- if total_users > 2:
	152	+ if total > 2:
150	153	print(f'{users[-1].id:>12} {users[-1].name}')
151		- print(f'Total: {total_users}.')
	154	+ print(f'Total: {total}.')
152	155
153	156
154	157	# ============================================================================
...	...	@@ -158,39 +161,45 @@ def main():
158	161	args = parse_commandline_arguments()
159	162
160	163	# --- database
161		- print(f'Using database: {args.db}')
162		- engine = sa.create_engine(f'sqlite:///{args.db}', echo=False)
	164	+ print(f'Database: {args.db}')
	165	+ engine = create_engine(f'sqlite:///{args.db}', echo=False, future=True)
163	166	Base.metadata.create_all(engine) # Criates schema if needed
164		- SessionMaker = sqlalchemy.orm.sessionmaker(bind=engine)
165		- session = SessionMaker()
	167	+ session = Session(engine, future=True)
166	168
167	169	# --- make list of students to insert
168		- new_students = []
	170	+ students = []
169	171
170	172	if args.admin:
171	173	print('Adding user: 0, Admin.')
172		- new_students.append({'uid': '0', 'name': 'Admin'})
	174	+ students.append({'uid': '0', 'name': 'Admin'})
173	175
174	176	for csvfile in args.csvfile:
175	177	print('Adding users from:', csvfile)
176		- new_students.extend(get_students_from_csv(csvfile))
	178	+ students.extend(get_students_from_csv(csvfile))
	179	+
	180	+ for csvfile in args.csvfile:
	181	+ print('Adding users from:', csvfile)
	182	+ students.extend(get_students_from_csv(csvfile))
177	183
178	184	if args.add:
179	185	for uid, name in args.add:
180	186	print(f'Adding user: {uid}, {name}.')
181		- new_students.append({'uid': uid, 'name': name})
	187	+ students.append({'uid': uid, 'name': name})
182	188
183	189	# --- insert new students
184		- if new_students:
	190	+ if students:
185	191	print('Generating password hashes', end='')
186		- with ThreadPoolExecutor() as executor: # hashing in parallel
187		- executor.map(lambda s: hashpw(s, args.pw), new_students)
188		- print(f'\nInserting {len(new_students)}')
189		- insert_students_into_db(session, new_students)
	192	+ with ThreadPoolExecutor() as executor: # hashing
	193	+ executor.map(lambda s: hashpw(s, args.pw), students)
	194	+ print(f'\nInserting {len(students)}')
	195	+ insert_students_into_db(session, students)
190	196
191	197	# --- update all students
192	198	if args.update_all:
193		- all_students = session.query(Student).filter(Student.id != '0').all()
	199	+ all_students = session.execute(
	200	+ select(Student).where(Student.id != '0')
	201	+ ).scalars().all()
	202	+
194	203	print(f'Updating password of {len(all_students)} users', end='')
195	204	for student in all_students:
196	205	password = (args.pw or student.id).encode('utf-8')
...	...	@@ -203,9 +212,12 @@ def main():
203	212	else:
204	213	for student_id in args.update:
205	214	print(f'Updating password of {student_id}')
206		- student = session.query(Student).get(student_id)
207		- password = (args.pw or student_id).encode('utf-8')
208		- student.password = bcrypt.hashpw(password, bcrypt.gensalt())
	215	+ student = session.execute(
	216	+ select(Student).
	217	+ where(Student.id == student_id)
	218	+ ).scalar_one()
	219	+ new_password = (args.pw or student_id).encode('utf-8')
	220	+ student.password = bcrypt.hashpw(new_password, bcrypt.gensalt())
209	221	session.commit()
210	222
211	223	show_students_in_database(session, args.verbose)
...	...
1	1	#!/usr/bin/env python3
2	2
3	3	'''
4		-Main file that starts the application and the web server
	4	+Start application and web server
5	5	'''
6	6
7	7
...	...	@@ -10,20 +10,17 @@ import argparse
10	10	import logging
11	11	import logging.config
12	12	import os
13		-from os import environ, path
14	13	import ssl
15	14	import sys
16		-# from typing import Any, Dict
17	15
18	16	# this project
19		-from perguntations.app import App, AppException
20		-from perguntations.serve import run_webserver
21		-from perguntations.tools import load_yaml
22		-from perguntations import APP_NAME, APP_VERSION
23		-
	17	+from .app import App, AppException
	18	+from .serve import run_webserver
	19	+from .tools import load_yaml
	20	+from . import APP_NAME, APP_VERSION
24	21
25	22	# ----------------------------------------------------------------------------
26		-def parse_cmdline_arguments():
	23	+def parse_cmdline_arguments() -> argparse.Namespace:
27	24	'''
28	25	Get command line arguments
29	26	'''
...	...	@@ -33,7 +30,6 @@ def parse_cmdline_arguments():
33	30	'included with this software before running the server.')
34	31	parser.add_argument('testfile',
35	32	type=str,
36		- # nargs='+', # TODO
37	33	help='tests in YAML format')
38	34	parser.add_argument('--allow-all',
39	35	action='store_true',
...	...	@@ -44,9 +40,6 @@ def parse_cmdline_arguments():
44	40	parser.add_argument('--debug',
45	41	action='store_true',
46	42	help='Enable debug messages')
47		- parser.add_argument('--show-ref',
48		- action='store_true',
49		- help='Show question references')
50	43	parser.add_argument('--review',
51	44	action='store_true',
52	45	help='Review mode: doesn\'t generate test')
...	...	@@ -63,56 +56,50 @@ def parse_cmdline_arguments():
63	56	help='Show version information and exit')
64	57	return parser.parse_args()
65	58
66		-
67	59	# ----------------------------------------------------------------------------
68	60	def get_logger_config(debug=False) -> dict:
69	61	'''
70	62	Load logger configuration from ~/.config directory if exists,
71	63	otherwise set default paramenters.
72	64	'''
	65	+
	66	+ file = 'logger-debug.yaml' if debug else 'logger.yaml'
	67	+ path = os.path.expanduser(os.environ.get('XDG_CONFIG_HOME', '~/.config/'))
	68	+ try:
	69	+ return load_yaml(os.path.join(path, APP_NAME, file))
	70	+ except OSError:
	71	+ print('Using default logger configuration...')
	72	+
73	73	if debug:
74		- filename = 'logger-debug.yaml'
75	74	level = 'DEBUG'
	75	+ fmt = '%(asctime)s %(levelname)-8s %(module)-12s%(lineno)4d\| %(message)s'
	76	+ dateformat = ''
76	77	else:
77		- filename = 'logger.yaml'
78	78	level = 'INFO'
79		-
80		- config_dir = environ.get('XDG_CONFIG_HOME', '~/.config/')
81		- config_file = path.join(path.expanduser(config_dir), APP_NAME, filename)
82		-
83		- default_config = {
84		- 'version': 1,
85		- 'formatters': {
86		- 'standard': {
87		- 'format': '%(asctime)s %(levelname)-8s %(message)s',
88		- 'datefmt': '%H:%M:%S',
89		- },
90		- },
91		- 'handlers': {
92		- 'default': {
93		- 'level': level,
94		- 'class': 'logging.StreamHandler',
95		- 'formatter': 'standard',
96		- 'stream': 'ext://sys.stdout',
	79	+ fmt = '%(asctime)s\| %(levelname)-8s\| %(message)s'
	80	+ dateformat = '%Y-%m-%d %H:%M:%S'
	81	+ modules = ['main', 'serve', 'app', 'models', 'questions', 'test',
	82	+ 'testfactory', 'tools']
	83	+ logger = {'handlers': ['default'], 'level': level, 'propagate': False}
	84	+ return {
	85	+ 'version': 1,
	86	+ 'formatters': {
	87	+ 'standard': {
	88	+ 'format': fmt,
	89	+ 'datefmt': dateformat,
	90	+ },
97	91	},
98		- },
99		- 'loggers': {
100		- '': { # configuration for serve.py
101		- 'handlers': ['default'],
102		- 'level': level,
	92	+ 'handlers': {
	93	+ 'default': {
	94	+ 'level': level,
	95	+ 'class': 'logging.StreamHandler',
	96	+ 'formatter': 'standard',
	97	+ 'stream': 'ext://sys.stdout',
	98	+ },
103	99	},
104		- },
	100	+ 'loggers': {f'{APP_NAME}.{module}': logger for module in modules}
105	101	}
106	102
107		- modules = ['app', 'models', 'questions', 'test', 'testfactory', 'tools']
108		- logger = {'handlers': ['default'], 'level': level, 'propagate': False}
109		-
110		- default_config['loggers'].update({f'{APP_NAME}.{module}': logger
111		- for module in modules})
112		-
113		- return load_yaml(config_file, default=default_config)
114		-
115		-
116	103	# ----------------------------------------------------------------------------
117	104	def main() -> None:
118	105	'''
...	...	@@ -122,15 +109,16 @@ def main() -> None:
122	109
123	110	# --- Setup logging ------------------------------------------------------
124	111	logging.config.dictConfig(get_logger_config(args.debug))
125		- logging.info('====================== Start Logging ======================')
	112	+ logger = logging.getLogger(__name__)
	113	+
	114	+ logger.info('================== Start Logging ==================')
126	115
127	116	# --- start application --------------------------------------------------
128	117	config = {
129	118	'testfile': args.testfile,
130		- 'debug': args.debug,
131	119	'allow_all': args.allow_all,
132	120	'allow_list': args.allow_list,
133		- 'show_ref': args.show_ref,
	121	+ 'debug': args.debug,
134	122	'review': args.review,
135	123	'correct': args.correct,
136	124	}
...	...	@@ -138,24 +126,24 @@ def main() -> None:
138	126	try:
139	127	app = App(config)
140	128	except AppException:
141		- logging.critical('Failed to start application.')
142		- sys.exit(-1)
	129	+ logger.critical('Failed to start application!')
	130	+ sys.exit(1)
143	131
144	132	# --- get SSL certificates -----------------------------------------------
145	133	if 'XDG_DATA_HOME' in os.environ:
146		- certs_dir = path.join(os.environ['XDG_DATA_HOME'], 'certs')
	134	+ certs_dir = os.path.join(os.environ['XDG_DATA_HOME'], 'certs')
147	135	else:
148		- certs_dir = path.expanduser('~/.local/share/certs')
	136	+ certs_dir = os.path.expanduser('~/.local/share/certs')
149	137
150	138	ssl_opt = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
151	139	try:
152		- ssl_opt.load_cert_chain(path.join(certs_dir, 'cert.pem'),
153		- path.join(certs_dir, 'privkey.pem'))
	140	+ ssl_opt.load_cert_chain(os.path.join(certs_dir, 'cert.pem'),
	141	+ os.path.join(certs_dir, 'privkey.pem'))
154	142	except FileNotFoundError:
155		- logging.critical('SSL certificates missing in %s', certs_dir)
156		- sys.exit(-1)
	143	+ logger.critical('SSL certificates missing in %s', certs_dir)
	144	+ sys.exit(1)
157	145
158		- # --- run webserver ----------------------------------------------------
	146	+ # --- run webserver ------------------------------------------------------
159	147	run_webserver(app=app, ssl_opt=ssl_opt, port=args.port, debug=args.debug)
160	148
161	149
...	...