Miguel Barão / aprendizations

Download as
Browse Code »

Commit a96cd2c78100a559e52fb5f5850fae498875f77b

Authored by Miguel Barão
1 parent 714897fd
Exists in master and in 1 other branch dev

- added logging system. 

- improved documentation.
- useful feedback when importing modules fails.
- markdown helper md() moved to tools.
- added empty certs folder (.gitignore)
Showing 7 changed files with 189 additions and 59 deletions   Show diff stats
BUGS.md
  Diff comments   View file @a96cd2c
1 1 BUGS:
2 2  
  3 +- se students.db não existe, rebenta.
  4 +- textarea tem codigo para preencher o texto, mas ja não é necessário porque pergunta não é reloaded.
3 5 - questions hardcoded in LearnApp.
4 6 - database hardcoded in LearnApp.
5 7 - implementar xsrf. Ver [http://www.tornadoweb.org/en/stable/guide/security.html#cross-site-request-forgery-protection]()
6 8  
7 9 TODO:
8 10  
  11 +- configuração e linha de comando.
  12 +- logging
9 13 - como gerar uma sequencia de perguntas?
10 14 - generators not working: bcrypt (ver blog)
11 15 - implementar navegacao radio/checkbox. cursor cima/baixo, espaco selecciona, enter submete.
... ...
README.md
  Diff comments   View file @a96cd2c
  1 +# Get Started
  2 +
  3 +
1 4 ## Requirements
2 5  
3   -You will need to install `python3.6`, `pip` and `sqlite3`.
4   -This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources. In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.6` or `pip-3.6`).
  6 +We will need to install python3.6, pip and sqlite3 python package.
  7 +This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources.
  8 +
  9 +- Installing from the system package management:
  10 + - OSX: `port install python36`
  11 + - FreeBSD: `pkg install python36 py36-sqlite3`
  12 + - Linux: `apt-get install ???`
  13 +- Installing from sources:
  14 + - Download from [http://www.python.org]()
  15 + - `unxz Python-3.6.tar.xz`
  16 + - `tar xvf Python-3.6.tar`
  17 + - `cd Python-3.6`
  18 + - `./configure --prefix=$HOME/.local/bin`
  19 + - `make && make install`
  20 +
  21 +To install pip (if not yet installed):
  22 +
  23 + python36 -m ensurepip --user
5 24  
6   -Install some python packages locally on the user area:
  25 +This will install pip in your account under `~/.local/bin`.
  26 +In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.6` or `pip-3.6` are used).
  27 +
  28 +Install additional python packages locally on the user area:
7 29  
8 30 pip install --user tornado sqlalchemy pyyaml pygments markdown bcrypt
9 31  
... ... @@ -12,6 +34,12 @@ These are usually installed under
12 34 - OSX: `~/Library/python/3.6/lib/python/site-packages/`
13 35 - Linux/FreeBSD: `~/.local/lib/python3.6/site-packages/`
14 36  
  37 +Note: If you want to always install python modules on the user account, edit the pip configuration file `~/.config/pip/pip.conf` (FreeBSD, Linux) or `Library/Application Support/pip/pip.conf` (OSX) and add the lines
  38 +
  39 + [global]
  40 + user = yes
  41 +
  42 +
15 43 ## Installation
16 44  
17 45 Replace USER by your bitbucket username:
... ... @@ -30,7 +58,7 @@ First we need to create a database:
30 58 ./initdb.py # initialize with a single user `0` and empty password
31 59 ./initdb.py --help # for the available options
32 60  
33   -We also need certificates for https. We can generate selfsigned certificates using openssl:
  61 +We also need certificates for https. Generate selfsigned certificates using openssl:
34 62  
35 63 mkdir certs
36 64 cd certs
... ...
app.py
  Diff comments   View file @a96cd2c
... ... @@ -2,16 +2,24 @@
2 2 import random
3 3 from contextlib import contextmanager # `with` statement in db sessions
4 4 from datetime import datetime
  5 +import logging
5 6  
6   -# libs
7   -import bcrypt
8   -from sqlalchemy import create_engine
9   -from sqlalchemy.orm import sessionmaker #, scoped_session
  7 +# user installed libraries
  8 +try:
  9 + import bcrypt
  10 + from sqlalchemy import create_engine
  11 + from sqlalchemy.orm import sessionmaker
  12 +except ImportError:
  13 + logger.critical('Python package missing. See README.md for instructions.')
  14 + sys.exit(1)
10 15  
11 16 # this project
12 17 import questions
13 18 from models import Student, Answer
14 19  
  20 +# setup logger for this module
  21 +logger = logging.getLogger(__name__)
  22 +
15 23 # ============================================================================
16 24 # LearnApp - application logic
17 25 # ============================================================================
... ... @@ -30,21 +38,23 @@ class LearnApp(object):
30 38 with self.db_session() as s:
31 39 n = s.query(Student).count() # filter(Student.id != '0').
32 40 except Exception as e:
33   - print('Database not usable.')
  41 + logger.critical('Database not usable.')
34 42 raise e
35 43 else:
36   - print('Database has {} students registered.'.format(n))
  44 + logger.info(f'Database has {n} students registered.')
37 45  
38 46 # ------------------------------------------------------------------------
39 47 def login(self, uid, try_pw):
40 48 with self.db_session() as s:
41 49 student = s.query(Student).filter(Student.id == uid).one_or_none()
42 50  
43   - if student is None or student in self.online: # FIXME
  51 + if student is None:
  52 + logger.info(f'User "{uid}" does not exist.')
44 53 return False # student does not exist or already loggeg in
45 54  
46 55 hashedtry = bcrypt.hashpw(try_pw.encode('utf-8'), student.password)
47 56 if hashedtry != student.password:
  57 + logger.info(f'User "{uid}" wrong password.')
48 58 return False # wrong password
49 59  
50 60 # success
... ...
certs/.gitignore 0 → 100644
  Diff comments   View file @a96cd2c
... ... @@ -0,0 +1,2 @@
  1 +*
  2 +!.gitignore
... ...
config/logger.yaml 0 → 100644
  Diff comments   View file @a96cd2c
... ... @@ -0,0 +1,36 @@
  1 +
  2 +version: 1
  3 +
  4 +formatters:
  5 + void:
  6 + format: ''
  7 + standard:
  8 + format: '%(asctime)s | %(levelname)-8s | %(name)-14s | %(message)s'
  9 +
  10 +handlers:
  11 + default:
  12 + level: 'INFO'
  13 + class: 'logging.StreamHandler'
  14 + formatter: 'standard'
  15 + stream: 'ext://sys.stdout'
  16 +
  17 +loggers:
  18 + '':
  19 + handlers: ['default']
  20 + level: 'INFO'
  21 +
  22 + 'app':
  23 + handlers: ['default']
  24 + level: 'INFO'
  25 + propagate: False
  26 +
  27 + 'questions':
  28 + handlers: ['default']
  29 + level: 'INFO'
  30 + propagate: False
  31 +
  32 + 'tools':
  33 + handlers: ['default']
  34 + level: 'INFO'
  35 + propagate: False
  36 +
... ...
serve.py
  Diff comments   View file @a96cd2c
... ... @@ -2,34 +2,30 @@
2 2  
3 3 # python standard library
4 4 import os
  5 +import sys
5 6 import json
6 7 import base64
7 8 import uuid
8   -
9   -# installed libraries
10   -import markdown
11   -import tornado.ioloop
12   -import tornado.web
13   -import tornado.httpserver
14   -from tornado import template, gen
15 9 import concurrent.futures
  10 +import logging.config
  11 +
  12 +# user installed libraries
  13 +try:
  14 + import markdown
  15 + import tornado.ioloop
  16 + import tornado.web
  17 + import tornado.httpserver
  18 + from tornado import template, gen
  19 +except ImportError:
  20 + print('Some python packages are missing. See README.md for instructions.')
  21 + sys.exit(1)
16 22  
17 23 # this project
18 24 from app import LearnApp
19   -
20   -# markdown helper
21   -def md(text):
22   - return markdown.markdown(text,
23   - extensions=[
24   - 'markdown.extensions.tables',
25   - 'markdown.extensions.fenced_code',
26   - 'markdown.extensions.codehilite',
27   - 'markdown.extensions.def_list',
28   - 'markdown.extensions.sane_lists'
29   - ])
  25 +from tools import load_yaml, md
30 26  
31 27 # A thread pool to be used for password hashing with bcrypt. FIXME and other things?
32   -executor = concurrent.futures.ThreadPoolExecutor(2)
  28 +# executor = concurrent.futures.ThreadPoolExecutor(2)
33 29  
34 30  
35 31 # ============================================================================
... ... @@ -89,11 +85,11 @@ class LoginHandler(BaseHandler):
89 85 # print(f'login.post: user={uid}, pw={pw}')
90 86  
91 87 if self.learn.login(uid, pw):
92   - print('login ok')
  88 + logging.info(f'User "{uid}" login ok.')
93 89 self.set_secure_cookie("user", str(uid), expires_days=30)
94 90 self.redirect(self.get_argument("next", "/"))
95 91 else:
96   - print('login failed')
  92 + logging.info(f'User "{uid}" login failed.')
97 93 self.render("login.html", error='Número ou senha incorrectos')
98 94  
99 95 # ----------------------------------------------------------------------------
... ... @@ -159,19 +155,40 @@ class QuestionHandler(BaseHandler):
159 155  
160 156 # ----------------------------------------------------------------------------
161 157 def main():
162   - webapp = WebApplication()
  158 + SERVER_PATH = os.path.dirname(os.path.realpath(__file__))
  159 + LOGGER_CONF = os.path.join(SERVER_PATH, 'config/logger.yaml')
  160 +
  161 +
  162 + # --- Setup logging
  163 + try:
  164 + logging.config.dictConfig(load_yaml(LOGGER_CONF))
  165 + except: # FIXME should this be done in a different way?
  166 + print('An error ocurred while setting up the logging system.')
  167 + print('Common causes:\n - inexistent directory "logs"?\n - write permission to "logs" directory?')
  168 + sys.exit(1)
  169 +
  170 + # --- start application
  171 + try:
  172 + webapp = WebApplication()
  173 + except:
  174 + logging.critical('Can\'t start application.')
  175 + sys.exit(1)
  176 +
  177 + # --- create webserver
163 178 http_server = tornado.httpserver.HTTPServer(webapp, ssl_options={
164 179 "certfile": "certs/cert.pem",
165 180 "keyfile": "certs/key.pem"
166 181 })
167 182 http_server.listen(8443)
168 183  
  184 + # --- start webserver
169 185 try:
170   - print('--- start ---')
  186 + logging.info('Webserver running...')
171 187 tornado.ioloop.IOLoop.current().start()
  188 + # running...
172 189 except KeyboardInterrupt:
173 190 tornado.ioloop.IOLoop.current().stop()
174   - print('\n--- stop ---')
  191 + logging.info('Webserver stopped.')
175 192  
176 193 # ----------------------------------------------------------------------------
177 194 if __name__ == "__main__":
... ...
tools.py
  Diff comments   View file @a96cd2c
... ... @@ -55,27 +55,60 @@ def run_script(script, stdin='', timeout=5):
55 55 else:
56 56 return output
57 57  
58   -def md_to_html(text, ref=None, files={}):
59   - if ref is not None:
60   - # given q['ref'] and q['files'] replaces references to files by a
61   - # GET to /file?ref=???;name=???
62   - for k in files:
63   - text = text.replace(k, '/file?ref={};name={}'.format(ref, k))
64   - return markdown.markdown(text, extensions=[
65   - 'markdown.extensions.tables',
66   - 'markdown.extensions.fenced_code',
67   - 'markdown.extensions.codehilite',
68   - 'markdown.extensions.def_list',
69   - 'markdown.extensions.sane_lists'
70   - ])
71   -
72   -def md_to_html_review(text, q):
73   - for k,f in q['files'].items():
74   - text = text.replace(k, '/absfile?name={}'.format(q['files'][k]))
75   - return markdown.markdown(text, extensions=[
76   - 'markdown.extensions.tables',
77   - 'markdown.extensions.fenced_code',
78   - 'markdown.extensions.codehilite',
79   - 'markdown.extensions.def_list',
80   - 'markdown.extensions.sane_lists'
81   - ])
  58 +
  59 +
  60 +# markdown helper
  61 +# returns a function md() that renders markdown with extensions
  62 +# this function is passed to templates for rendering
  63 +def md(text):
  64 + return markdown.markdown(text,
  65 + extensions=[
  66 + 'markdown.extensions.tables',
  67 + 'markdown.extensions.fenced_code',
  68 + 'markdown.extensions.codehilite',
  69 + 'markdown.extensions.def_list',
  70 + 'markdown.extensions.sane_lists'
  71 + ])
  72 +
  73 +
  74 +
  75 +
  76 +
  77 +
  78 +
  79 +
  80 +
  81 +
  82 +
  83 +
  84 +
  85 +
  86 +
  87 +
  88 +
  89 +
  90 +
  91 +# def md_to_html(text, ref=None, files={}):
  92 +# if ref is not None:
  93 +# # given q['ref'] and q['files'] replaces references to files by a
  94 +# # GET to /file?ref=???;name=???
  95 +# for k in files:
  96 +# text = text.replace(k, '/file?ref={};name={}'.format(ref, k))
  97 +# return markdown.markdown(text, extensions=[
  98 +# 'markdown.extensions.tables',
  99 +# 'markdown.extensions.fenced_code',
  100 +# 'markdown.extensions.codehilite',
  101 +# 'markdown.extensions.def_list',
  102 +# 'markdown.extensions.sane_lists'
  103 +# ])
  104 +
  105 +# def md_to_html_review(text, q):
  106 +# for k,f in q['files'].items():
  107 +# text = text.replace(k, '/absfile?name={}'.format(q['files'][k]))
  108 +# return markdown.markdown(text, extensions=[
  109 +# 'markdown.extensions.tables',
  110 +# 'markdown.extensions.fenced_code',
  111 +# 'markdown.extensions.codehilite',
  112 +# 'markdown.extensions.def_list',
  113 +# 'markdown.extensions.sane_lists'
  114 +# ])
... ...