Miguel Barão / aprendizations

Download as
Browse Code »

Commit a96cd2c78100a559e52fb5f5850fae498875f77b

Authored by Miguel Barão
1 parent 714897fd
Exists in master and in 1 other branch dev

- added logging system. 

- improved documentation.
- useful feedback when importing modules fails.
- markdown helper md() moved to tools.
- added empty certs folder (.gitignore)
Showing 7 changed files with 189 additions and 59 deletions   Show diff stats
1 BUGS: 1 BUGS:
2 2
  3 +- se students.db não existe, rebenta.
  4 +- textarea tem codigo para preencher o texto, mas ja não é necessário porque pergunta não é reloaded.
3 - questions hardcoded in LearnApp. 5 - questions hardcoded in LearnApp.
4 - database hardcoded in LearnApp. 6 - database hardcoded in LearnApp.
5 - implementar xsrf. Ver [http://www.tornadoweb.org/en/stable/guide/security.html#cross-site-request-forgery-protection]() 7 - implementar xsrf. Ver [http://www.tornadoweb.org/en/stable/guide/security.html#cross-site-request-forgery-protection]()
6 8
7 TODO: 9 TODO:
8 10
  11 +- configuração e linha de comando.
  12 +- logging
9 - como gerar uma sequencia de perguntas? 13 - como gerar uma sequencia de perguntas?
10 - generators not working: bcrypt (ver blog) 14 - generators not working: bcrypt (ver blog)
11 - implementar navegacao radio/checkbox. cursor cima/baixo, espaco selecciona, enter submete. 15 - implementar navegacao radio/checkbox. cursor cima/baixo, espaco selecciona, enter submete.
  1 +# Get Started
  2 +
  3 +
1 ## Requirements 4 ## Requirements
2 5
3 -You will need to install `python3.6`, `pip` and `sqlite3`.  
4 -This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources. In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.6` or `pip-3.6`). 6 +We will need to install python3.6, pip and sqlite3 python package.
  7 +This can be done using the system package management, downloaded from [http://www.python.org](), or compiled from sources.
  8 +
  9 +- Installing from the system package management:
  10 + - OSX: `port install python36`
  11 + - FreeBSD: `pkg install python36 py36-sqlite3`
  12 + - Linux: `apt-get install ???`
  13 +- Installing from sources:
  14 + - Download from [http://www.python.org]()
  15 + - `unxz Python-3.6.tar.xz`
  16 + - `tar xvf Python-3.6.tar`
  17 + - `cd Python-3.6`
  18 + - `./configure --prefix=$HOME/.local/bin`
  19 + - `make && make install`
  20 +
  21 +To install pip (if not yet installed):
  22 +
  23 + python36 -m ensurepip --user
5 24
6 -Install some python packages locally on the user area: 25 +This will install pip in your account under `~/.local/bin`.
  26 +In the end you should be able to run `pip3 --version` and `python3 -c "import sqlite3"` without errors (sometimes `pip3` is `pip`, `pip3.6` or `pip-3.6` are used).
  27 +
  28 +Install additional python packages locally on the user area:
7 29
8 pip install --user tornado sqlalchemy pyyaml pygments markdown bcrypt 30 pip install --user tornado sqlalchemy pyyaml pygments markdown bcrypt
9 31
@@ -12,6 +34,12 @@ These are usually installed under @@ -12,6 +34,12 @@ These are usually installed under
12 - OSX: `~/Library/python/3.6/lib/python/site-packages/` 34 - OSX: `~/Library/python/3.6/lib/python/site-packages/`
13 - Linux/FreeBSD: `~/.local/lib/python3.6/site-packages/` 35 - Linux/FreeBSD: `~/.local/lib/python3.6/site-packages/`
14 36
  37 +Note: If you want to always install python modules on the user account, edit the pip configuration file `~/.config/pip/pip.conf` (FreeBSD, Linux) or `Library/Application Support/pip/pip.conf` (OSX) and add the lines
  38 +
  39 + [global]
  40 + user = yes
  41 +
  42 +
15 ## Installation 43 ## Installation
16 44
17 Replace USER by your bitbucket username: 45 Replace USER by your bitbucket username:
@@ -30,7 +58,7 @@ First we need to create a database: @@ -30,7 +58,7 @@ First we need to create a database:
30 ./initdb.py # initialize with a single user `0` and empty password 58 ./initdb.py # initialize with a single user `0` and empty password
31 ./initdb.py --help # for the available options 59 ./initdb.py --help # for the available options
32 60
33 -We also need certificates for https. We can generate selfsigned certificates using openssl: 61 +We also need certificates for https. Generate selfsigned certificates using openssl:
34 62
35 mkdir certs 63 mkdir certs
36 cd certs 64 cd certs
@@ -2,16 +2,24 @@ @@ -2,16 +2,24 @@
2 import random 2 import random
3 from contextlib import contextmanager # `with` statement in db sessions 3 from contextlib import contextmanager # `with` statement in db sessions
4 from datetime import datetime 4 from datetime import datetime
  5 +import logging
5 6
6 -# libs  
7 -import bcrypt  
8 -from sqlalchemy import create_engine  
9 -from sqlalchemy.orm import sessionmaker #, scoped_session 7 +# user installed libraries
  8 +try:
  9 + import bcrypt
  10 + from sqlalchemy import create_engine
  11 + from sqlalchemy.orm import sessionmaker
  12 +except ImportError:
  13 + logger.critical('Python package missing. See README.md for instructions.')
  14 + sys.exit(1)
10 15
11 # this project 16 # this project
12 import questions 17 import questions
13 from models import Student, Answer 18 from models import Student, Answer
14 19
  20 +# setup logger for this module
  21 +logger = logging.getLogger(__name__)
  22 +
15 # ============================================================================ 23 # ============================================================================
16 # LearnApp - application logic 24 # LearnApp - application logic
17 # ============================================================================ 25 # ============================================================================
@@ -30,21 +38,23 @@ class LearnApp(object): @@ -30,21 +38,23 @@ class LearnApp(object):
30 with self.db_session() as s: 38 with self.db_session() as s:
31 n = s.query(Student).count() # filter(Student.id != '0'). 39 n = s.query(Student).count() # filter(Student.id != '0').
32 except Exception as e: 40 except Exception as e:
33 - print('Database not usable.') 41 + logger.critical('Database not usable.')
34 raise e 42 raise e
35 else: 43 else:
36 - print('Database has {} students registered.'.format(n)) 44 + logger.info(f'Database has {n} students registered.')
37 45
38 # ------------------------------------------------------------------------ 46 # ------------------------------------------------------------------------
39 def login(self, uid, try_pw): 47 def login(self, uid, try_pw):
40 with self.db_session() as s: 48 with self.db_session() as s:
41 student = s.query(Student).filter(Student.id == uid).one_or_none() 49 student = s.query(Student).filter(Student.id == uid).one_or_none()
42 50
43 - if student is None or student in self.online: # FIXME 51 + if student is None:
  52 + logger.info(f'User "{uid}" does not exist.')
44 return False # student does not exist or already loggeg in 53 return False # student does not exist or already loggeg in
45 54
46 hashedtry = bcrypt.hashpw(try_pw.encode('utf-8'), student.password) 55 hashedtry = bcrypt.hashpw(try_pw.encode('utf-8'), student.password)
47 if hashedtry != student.password: 56 if hashedtry != student.password:
  57 + logger.info(f'User "{uid}" wrong password.')
48 return False # wrong password 58 return False # wrong password
49 59
50 # success 60 # success
certs/.gitignore 0 → 100644
  Diff comments   View file @a96cd2c
@@ -0,0 +1,2 @@ @@ -0,0 +1,2 @@
  1 +*
  2 +!.gitignore
config/logger.yaml 0 → 100644
  Diff comments   View file @a96cd2c
@@ -0,0 +1,36 @@ @@ -0,0 +1,36 @@
  1 +
  2 +version: 1
  3 +
  4 +formatters:
  5 + void:
  6 + format: ''
  7 + standard:
  8 + format: '%(asctime)s | %(levelname)-8s | %(name)-14s | %(message)s'
  9 +
  10 +handlers:
  11 + default:
  12 + level: 'INFO'
  13 + class: 'logging.StreamHandler'
  14 + formatter: 'standard'
  15 + stream: 'ext://sys.stdout'
  16 +
  17 +loggers:
  18 + '':
  19 + handlers: ['default']
  20 + level: 'INFO'
  21 +
  22 + 'app':
  23 + handlers: ['default']
  24 + level: 'INFO'
  25 + propagate: False
  26 +
  27 + 'questions':
  28 + handlers: ['default']
  29 + level: 'INFO'
  30 + propagate: False
  31 +
  32 + 'tools':
  33 + handlers: ['default']
  34 + level: 'INFO'
  35 + propagate: False
  36 +
@@ -2,34 +2,30 @@ @@ -2,34 +2,30 @@
2 2
3 # python standard library 3 # python standard library
4 import os 4 import os
  5 +import sys
5 import json 6 import json
6 import base64 7 import base64
7 import uuid 8 import uuid
8 -  
9 -# installed libraries  
10 -import markdown  
11 -import tornado.ioloop  
12 -import tornado.web  
13 -import tornado.httpserver  
14 -from tornado import template, gen  
15 import concurrent.futures 9 import concurrent.futures
  10 +import logging.config
  11 +
  12 +# user installed libraries
  13 +try:
  14 + import markdown
  15 + import tornado.ioloop
  16 + import tornado.web
  17 + import tornado.httpserver
  18 + from tornado import template, gen
  19 +except ImportError:
  20 + print('Some python packages are missing. See README.md for instructions.')
  21 + sys.exit(1)
16 22
17 # this project 23 # this project
18 from app import LearnApp 24 from app import LearnApp
19 -  
20 -# markdown helper  
21 -def md(text):  
22 - return markdown.markdown(text,  
23 - extensions=[  
24 - 'markdown.extensions.tables',  
25 - 'markdown.extensions.fenced_code',  
26 - 'markdown.extensions.codehilite',  
27 - 'markdown.extensions.def_list',  
28 - 'markdown.extensions.sane_lists'  
29 - ]) 25 +from tools import load_yaml, md
30 26
31 # A thread pool to be used for password hashing with bcrypt. FIXME and other things? 27 # A thread pool to be used for password hashing with bcrypt. FIXME and other things?
32 -executor = concurrent.futures.ThreadPoolExecutor(2) 28 +# executor = concurrent.futures.ThreadPoolExecutor(2)
33 29
34 30
35 # ============================================================================ 31 # ============================================================================
@@ -89,11 +85,11 @@ class LoginHandler(BaseHandler): @@ -89,11 +85,11 @@ class LoginHandler(BaseHandler):
89 # print(f'login.post: user={uid}, pw={pw}') 85 # print(f'login.post: user={uid}, pw={pw}')
90 86
91 if self.learn.login(uid, pw): 87 if self.learn.login(uid, pw):
92 - print('login ok') 88 + logging.info(f'User "{uid}" login ok.')
93 self.set_secure_cookie("user", str(uid), expires_days=30) 89 self.set_secure_cookie("user", str(uid), expires_days=30)
94 self.redirect(self.get_argument("next", "/")) 90 self.redirect(self.get_argument("next", "/"))
95 else: 91 else:
96 - print('login failed') 92 + logging.info(f'User "{uid}" login failed.')
97 self.render("login.html", error='Número ou senha incorrectos') 93 self.render("login.html", error='Número ou senha incorrectos')
98 94
99 # ---------------------------------------------------------------------------- 95 # ----------------------------------------------------------------------------
@@ -159,19 +155,40 @@ class QuestionHandler(BaseHandler): @@ -159,19 +155,40 @@ class QuestionHandler(BaseHandler):
159 155
160 # ---------------------------------------------------------------------------- 156 # ----------------------------------------------------------------------------
161 def main(): 157 def main():
162 - webapp = WebApplication() 158 + SERVER_PATH = os.path.dirname(os.path.realpath(__file__))
  159 + LOGGER_CONF = os.path.join(SERVER_PATH, 'config/logger.yaml')
  160 +
  161 +
  162 + # --- Setup logging
  163 + try:
  164 + logging.config.dictConfig(load_yaml(LOGGER_CONF))
  165 + except: # FIXME should this be done in a different way?
  166 + print('An error ocurred while setting up the logging system.')
  167 + print('Common causes:\n - inexistent directory "logs"?\n - write permission to "logs" directory?')
  168 + sys.exit(1)
  169 +
  170 + # --- start application
  171 + try:
  172 + webapp = WebApplication()
  173 + except:
  174 + logging.critical('Can\'t start application.')
  175 + sys.exit(1)
  176 +
  177 + # --- create webserver
163 http_server = tornado.httpserver.HTTPServer(webapp, ssl_options={ 178 http_server = tornado.httpserver.HTTPServer(webapp, ssl_options={
164 "certfile": "certs/cert.pem", 179 "certfile": "certs/cert.pem",
165 "keyfile": "certs/key.pem" 180 "keyfile": "certs/key.pem"
166 }) 181 })
167 http_server.listen(8443) 182 http_server.listen(8443)
168 183
  184 + # --- start webserver
169 try: 185 try:
170 - print('--- start ---') 186 + logging.info('Webserver running...')
171 tornado.ioloop.IOLoop.current().start() 187 tornado.ioloop.IOLoop.current().start()
  188 + # running...
172 except KeyboardInterrupt: 189 except KeyboardInterrupt:
173 tornado.ioloop.IOLoop.current().stop() 190 tornado.ioloop.IOLoop.current().stop()
174 - print('\n--- stop ---') 191 + logging.info('Webserver stopped.')
175 192
176 # ---------------------------------------------------------------------------- 193 # ----------------------------------------------------------------------------
177 if __name__ == "__main__": 194 if __name__ == "__main__":
@@ -55,27 +55,60 @@ def run_script(script, stdin='', timeout=5): @@ -55,27 +55,60 @@ def run_script(script, stdin='', timeout=5):
55 else: 55 else:
56 return output 56 return output
57 57
58 -def md_to_html(text, ref=None, files={}):  
59 - if ref is not None:  
60 - # given q['ref'] and q['files'] replaces references to files by a  
61 - # GET to /file?ref=???;name=???  
62 - for k in files:  
63 - text = text.replace(k, '/file?ref={};name={}'.format(ref, k))  
64 - return markdown.markdown(text, extensions=[  
65 - 'markdown.extensions.tables',  
66 - 'markdown.extensions.fenced_code',  
67 - 'markdown.extensions.codehilite',  
68 - 'markdown.extensions.def_list',  
69 - 'markdown.extensions.sane_lists'  
70 - ])  
71 -  
72 -def md_to_html_review(text, q):  
73 - for k,f in q['files'].items():  
74 - text = text.replace(k, '/absfile?name={}'.format(q['files'][k]))  
75 - return markdown.markdown(text, extensions=[  
76 - 'markdown.extensions.tables',  
77 - 'markdown.extensions.fenced_code',  
78 - 'markdown.extensions.codehilite',  
79 - 'markdown.extensions.def_list',  
80 - 'markdown.extensions.sane_lists'  
81 - ]) 58 +
  59 +
  60 +# markdown helper
  61 +# returns a function md() that renders markdown with extensions
  62 +# this function is passed to templates for rendering
  63 +def md(text):
  64 + return markdown.markdown(text,
  65 + extensions=[
  66 + 'markdown.extensions.tables',
  67 + 'markdown.extensions.fenced_code',
  68 + 'markdown.extensions.codehilite',
  69 + 'markdown.extensions.def_list',
  70 + 'markdown.extensions.sane_lists'
  71 + ])
  72 +
  73 +
  74 +
  75 +
  76 +
  77 +
  78 +
  79 +
  80 +
  81 +
  82 +
  83 +
  84 +
  85 +
  86 +
  87 +
  88 +
  89 +
  90 +
  91 +# def md_to_html(text, ref=None, files={}):
  92 +# if ref is not None:
  93 +# # given q['ref'] and q['files'] replaces references to files by a
  94 +# # GET to /file?ref=???;name=???
  95 +# for k in files:
  96 +# text = text.replace(k, '/file?ref={};name={}'.format(ref, k))
  97 +# return markdown.markdown(text, extensions=[
  98 +# 'markdown.extensions.tables',
  99 +# 'markdown.extensions.fenced_code',
  100 +# 'markdown.extensions.codehilite',
  101 +# 'markdown.extensions.def_list',
  102 +# 'markdown.extensions.sane_lists'
  103 +# ])
  104 +
  105 +# def md_to_html_review(text, q):
  106 +# for k,f in q['files'].items():
  107 +# text = text.replace(k, '/absfile?name={}'.format(q['files'][k]))
  108 +# return markdown.markdown(text, extensions=[
  109 +# 'markdown.extensions.tables',
  110 +# 'markdown.extensions.fenced_code',
  111 +# 'markdown.extensions.codehilite',
  112 +# 'markdown.extensions.def_list',
  113 +# 'markdown.extensions.sane_lists'
  114 +# ])