- corrections in the MANUAL.md

- corrections in the /config/server.conf - fixed paths so that server can be run from other directories. - fixed paths so that questions, tests and logs do not have to be in the server directory. - check existence of some files, directories and database - removed offensive comments

- corrections in the MANUAL.md
- corrections in the /config/server.conf - fixed paths so that server can be run from other directories. - fixed paths so that questions, tests and logs do not have to be in the server directory. - check existence of some files, directories and database - removed offensive comments
Miguel Barão
1 parent 34b58efe
Showing 13 changed files with 122 additions and 118 deletions Show diff stats
BUGS.md
MANUAL.md
conf/server.conf
demo/questions.yaml
demo/test.yaml
logs/.gitignore
myauth.py
questions.py
serve.py
templates/grade.html
templates/login.html
templates/test.html
test.py
@@ -2,10 +2,10 @@
 # BUGS
-!!! - questions type script, necessário dar um caminho exacto relativamete ao directorio do server em vez da pergunta. deveria ser possivel mover as perguntas de directorio sem rebentar os caminhos.
-
+- check if script to generate questions exist before instantiation.
+- paths manipulation in strings is unix only ('/something'). use os.path to create paths.
+- fix ans directory. relative to what?? current dir?
 - parece que é preciso criar à mão a pasta para as respostas (ans/...) depois apercebo-me que os caminhos no teste dizem respeito à directoria donde o teste é corrido... as respostas deveriam guardadas no directório dado.
-- se database for mal configurada, é criada uma base de dados vazia e rebenta na autenticacao.
 - testar regex na definicao das perguntas. como se faz rawstring em yaml? singlequote? problemas de backslash??? sim... necessário fazer \\ em varios casos, mas não é claro! e.g. \n é convertido em espaço mas \w é convertido em \\ e w.
 # TODO
@@ -25,6 +25,10 @@
 # FIXED
+- se database for mal configurada, é criada uma base de dados vazia e rebenta na autenticacao.
+- questions type script, necessário dar um caminho exacto relativamete ao directorio do server em vez da pergunta. deveria ser possivel mover as perguntas de directorio sem rebentar os caminhos.
+- check that files exist in questions generator e correct textarea. add path in test.yaml
+- scripts generator and correct should consider the questions path.
 - testar envio de parametros para stdin para perguntas tipo generator.
 - mathjax e jquery no login
 - mostrar erro quando nao consegue importar questions files
@@ -11,10 +11,27 @@ Install python 3.4 and the following packages from pip:
 Before using the program you need to
-1. Create the students database
-1. Create questions
-1. Create a test
-1. Configure the server (the default should be enough)
+1. Edit `config/server.conf` in the server directory and define
+    - Logging
+
+        `log.error_file= '/Users/USERNAME/Library/Logs/Perguntations/errors.log'`
+
+        `log.access_file= '/Users/USERNAME/Library/Logs/Perguntations/access.log'`
+
+        You must create the directories if they do not exist already.
+
+        Setting these locations to empty strings `''` disables logging.
+
+    - Sessions
+        If `tools.sessions.storage_type='file'` sessions are saved on the file system in the location given in `tools.sessions.storage_path`. Restarting the server will maintain the sessions active.
+
+        If `storage_type='ram'` (default) no files are stored but restaring the server will reset sessions.
+
+        You should give enough time in the `tools.sessions.timeout` to complete an exam. The default is 240 minutes (4 hours).
+
+1. Create the students database (see below)
+1. Create questions (see below)
+1. Create a test (see below)
 ### Create students database
@@ -103,13 +120,17 @@ debug: False
 # Show the file and ref field of each question
 show_ref: True
-# -------------------------------------------------------------------------
-# This are the questions database to be imported.
+# ----------------------------------------------------------------------------
+# Location of the questions files (absolute path or relative to current dir)
+path: questions
+
+# This are the questions files to be imported.
 files:
-    - questions/file1.yaml
-    - questions/file2.yaml
-    - questions/file3.yaml
-# -------------------------------------------------------------------------
+    - file1.yaml
+    - file2.yaml
+    - file3.yaml
+
+# ----------------------------------------------------------------------------
 # This is the actual test configuration. Selection of questions and points
 # It'a defined as a list of questions. Each question can be a single
 # question key or a list of keys from which one is chosen at random.
@@ -129,7 +150,7 @@ questions:
 This following one is wrong:
     -   wrong-question          # missing "ref:" key
-        points: 2               
+        points: 2
 ```
 Some of the options have default values if they are omitted. The defaults are the following:
@@ -265,9 +286,10 @@ The server will try to convert the printed message to a float, a failure will gi
     ref: some-key
     type: textarea
     text: write an expression to add x and y.    # optional (default: '')
-    correct: path/to/myscript
+    correct: myscript
 ```
+The script location is the same as the questions file.
 An example of a script in python that validades an answer is
 ```python
@@ -1,37 +0,0 @@
-# -*- coding: utf-8 -*-
-
-[global]
-;environment= 'production'
-
-; number of threads running
-server.thread_pool= 10
-
-; Host address and port
-; set socket_port = 443 if SSL is enabled below, 8080 otherwise
-; if port is 443 then the server should be run as root. SSL also works on 8080.
-server.socket_host = '0.0.0.0'
-server.socket_port = 8080
-
-; Uncomment to enable SSL (see README.txt on how to generate certificates)
-; server.ssl_module can be 'builtin' or 'pyopenssl'.
-; server.ssl_module = 'builtin'
-; server.ssl_certificate = 'certs/webserver.crt'
-; server.ssl_private_key = 'certs/webserver.key'
-; not required for snakeoil:
-; server.ssl_certificate_chain = 'ca_certs.crt'
-
-log.screen = False
-log.error_file = 'logs/errors.log'
-log.access_file = 'logs/access.log'
-
-tools.sessions.on = True
-tools.sessions.timeout = 240
-tools.sessions.storage_type = 'ram'
-tools.sessions.storage_path = 'sessions'
-tools.auth.on = True
-
-[/]
-tools.staticdir.root = os.path.normpath(os.path.abspath(os.path.curdir))
-tools.staticdir.dir = 'static'
-tools.staticdir.on = True
-; tools.staticdir.debug = True
@@ -44,13 +44,13 @@
     ref: question-colors
     type: textarea
     text: Write names of the three basic colors.
-    correct: demo/correct-question.py
+    correct: correct-question.py
     hint: They start by RGB and order does not matter.
 # ---------------------------------------------------------------------------
 -
     ref: question-whatever
     type: generator
-    script: demo/generate-question.py
+    script: generate-question.py
     arg: "11,120"
     # the script should print a question in yaml format like the ones above.
     # Print only the dictionary, not the list (hiffen).
@@ -4,7 +4,7 @@ title: Teste de Demonstração
 # database contains students+passwords, final results of the tests, and questions done
 # database: path/to/students.db
-database: db/students.db
+database: inscritos.db
 # this will generate a file for each test done. The file is like a replacement
 # for a test done in paper.
@@ -16,19 +16,20 @@ show_hints: True
 practice: True
 # debug: False
-# When in practice mode and an answer is wrong (i.e. less that 0.5 correct),
-# an insult is chosen from the list (optional)
-offensive:
-    - Ó meu grande asno, então não sabes esta?
-    - Pois, pois... não estudes não...
-    - E eu sou o Elvis Presley...
-    - Pois, e bróculos também.
+#-----------------------------------------------------------------------------
+# This is the base path applied to the questions files and all the scripts
+# including generators and correctors.
+# Either absolute path or relative to current directory.
+path: demo
+# (Note: answers are saved in a different path defined in answers_dir)
 #-----------------------------------------------------------------------------
 # List of files containing questions in yaml format.
 # Selected questions will be obtained from these files.
+# (search in absolute path or current working directory)
 files:
-    - demo/questions.yaml
+    - questions.yaml
+
 #-----------------------------------------------------------------------------
 # This is the list of questions. If a "ref:" has a list of keys, then
 # one question is selected from the list.
@@ -1,3 +0,0 @@
-# ignore everything except .gitignore
-*
-!.gitignore
 \ No newline at end of file
@@ -11,10 +11,14 @@ from hashlib import sha256
 from mako.lookup import TemplateLookup
 import urllib
 import html
+from os import path
+
+# path where this file is located
+server_path = path.dirname(path.realpath(__file__))
 SESSION_KEY = 'userid'
-templates = TemplateLookup(directories=['templates'], input_encoding='utf-8')
+templates = TemplateLookup(directories=[server_path+'/templates'], input_encoding='utf-8')
 def credentials_ok(uid, password, db):
@@ -152,7 +156,7 @@ class AuthController(object):
                 cherrypy.session[SESSION_KEY] = cherrypy.request.login = uid
                 cherrypy.session['name'] = name
                 raise cherrypy.HTTPRedirect(from_page)
-        logintemplate = templates.get_template('login.html')
+        logintemplate = templates.get_template('/login.html')
         return logintemplate.render(from_page=from_page)
     @cherrypy.expose
@@ -4,6 +4,7 @@ import random
 import re
 import subprocess
 import sys
+import os.path
 # Example usage:
 #
@@ -18,19 +19,19 @@ import sys
 #   test[0]['answer'] = 42           # insert answer
 #   grade = test[0].correct()        # correct answer
-
 # ===========================================================================
 class QuestionsPool(dict):
     '''This class contains base questions read from files, but which are
     not ready yet. They have to be instantiated separatly for each student.'''
     #------------------------------------------------------------------------
-    def add_from_file(self, filename):
+    def add_from_file(self, filename, path='.'):
+        path_file = os.path.normpath(os.path.join(path, filename))
         try:
-            with open(filename, 'r') as f:
+            with open(path_file, 'r') as f:
                 questions = yaml.load(f)
         except(FileNotFoundError):
-            print('  * Questions file "{0}" not found. Aborting...'.format(filename))
+            print('  * Questions file "{0}" not found. Aborting...'.format(path_file))
             sys.exit(1)
         except(yaml.parser.ParserError):
             print('  * Error in questions file "{0}". Aborting...'.format(filename))
@@ -47,6 +48,7 @@ class QuestionsPool(dict):
             # filename and index (number in the file, 0 based)
             q['filename'] = filename
+            q['path'] = path
             q['index'] = i
             # ref  (if missing, add 'filename.yaml:3')
@@ -59,9 +61,9 @@ class QuestionsPool(dict):
             self[q['ref']] = q
     #------------------------------------------------------------------------
-    def add_from_files(self, file_list):
-        for filename in file_list:
-            self.add_from_file(filename)
+    def add_from_files(self, files, path='.'):
+        for filename in files:
+            self.add_from_file(filename, path)
 #============================================================================
@@ -116,10 +118,12 @@ def question_generator(q):
     q['arg'] = q.get('arg', '')  # send this string to stdin
+    script = os.path.abspath(os.path.normpath(os.path.join(q['path'], q['script'])))
     try:
-        p = subprocess.Popen([q['script']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+        p = subprocess.Popen([script], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
     except FileNotFoundError:
-        print('  * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(q['script'], q['ref'], q['filename']))
+        print('  * Script "{0}" of question "{1}" in file "{2}" could not be found'.format(script, q['ref'], q['filename']))
+        sys.exit(1)
     try:
         qyaml = p.communicate(input=q['arg'].encode('utf-8'), timeout=5)[0].decode('utf-8')
@@ -386,8 +390,9 @@ class QuestionTextArea(Question):
             # The correction program expects data from stdin and prints the result to stdout.
             # The result should be a string that can be parsed to a float.
+            script = os.path.abspath(os.path.normpath(os.path.join(self['path'], self['correct'])))
             try:
-                p = subprocess.Popen([self['correct']], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
+                p = subprocess.Popen([script], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.STDOUT)
             except FileNotFoundError as e:
                 print('  * Script "{0}" defined in question "{1}" of file "{2}" could not be found'.format(self['correct'], self['ref'], self['filename']))
                 raise e
@@ -7,12 +7,19 @@
 import cherrypy
 from mako.lookup import TemplateLookup
 import argparse
+from os import path
+
+# path where this file is located
+server_path = path.dirname(path.realpath(__file__))
 # my code
 from myauth import AuthController, require
 import test
 import database
+TEMPLATES_DIR = server_path + '/templates'
+
+
 # ============================================================================
 #   Classes that respond to HTTP
 # ============================================================================
@@ -22,7 +29,7 @@ class Root(object):
         self.testconf = testconf  # base test dict (not instance)
         self.database = database.Database(testconf['database'])
         self.auth = AuthController(database=testconf['database'])
-        self.templates = TemplateLookup(directories=['templates'], input_encoding='utf-8')
+        self.templates = TemplateLookup(directories=[TEMPLATES_DIR], input_encoding='utf-8')
         self.tags = {'online': set(), 'finished': set()}  # FIXME should be in application, not server
     # --- DEFAULT ------------------------------------------------------------
@@ -32,7 +39,7 @@ class Root(object):
     def default(self, *args):
         raise cherrypy.HTTPRedirect('/test')
-    # --- LOGOUT ------------------------------------------------------------
+    # --- LOGOUT -------------------------------------------------------------
     @cherrypy.expose
     @require()
     def logout(self):
@@ -57,7 +64,7 @@ class Root(object):
                 cherrypy.log.error('Password updated for student %s.' % str(num), 'APPLICATION')
         students = self.database.get_students()
-        template = self.templates.get_template('students.html')
+        template = self.templates.get_template('/students.html')
         return template.render(students=students, tags=self.tags)
     # --- RESULTS ------------------------------------------------------------
@@ -65,7 +72,7 @@ class Root(object):
     def results(self):
         if self.testconf.get('practice', False):
             r = self.database.test_grades(self.testconf['ref'])
-            template = self.templates.get_template('results.html')
+            template = self.templates.get_template('/results.html')
             return template.render(t=self.testconf, results=r)
         else:
             raise cherrypy.HTTPRedirect('/')
@@ -88,7 +95,7 @@ class Root(object):
             self.tags['online'].add(uid)  # track logged in students
         # Generate question
-        template = self.templates.get_template('test.html')
+        template = self.templates.get_template('/test.html')
         return template.render(t=t, questions=t['questions'])
     # --- CORRECT ------------------------------------------------------------
@@ -146,7 +153,8 @@ class Root(object):
 # ============================================================================
 def parse_arguments():
     argparser = argparse.ArgumentParser(description='Server for online tests. Enrolled students and tests have to be previously configured. Please read the documentation included with this software before running the server.')
-    argparser.add_argument('--server', default='conf/server.conf', type=str, help='server configuration file')
+    serverconf_file = path.normpath(path.join(server_path, 'config', 'server.conf'))
+    argparser.add_argument('--server', default=serverconf_file, type=str, help='server configuration file')
     argparser.add_argument('--debug', action='store_true',
         help='Show datastructures when rendering questions')
     argparser.add_argument('--show_ref', action='store_true',
@@ -159,7 +167,7 @@ def parse_arguments():
         help='Saves answers in JSON format')
     argparser.add_argument('--practice', action='store_true',
         help='Show correction results and allow repetitive resubmission of the test')
-    argparser.add_argument('testfile', type=str, nargs='+', help='test in YAML format.')
+    argparser.add_argument('testfile', type=str, nargs='+', help='test/exam in YAML format.') # FIXME only one exam supported at the moment
     return argparser.parse_args()
 # ============================================================================
@@ -180,7 +188,8 @@ if __name__ == &#39;__main__&#39;:
     root = Root(testconf)
     # --- Mount and run server.
+    cherrypy.config.update({'tools.staticdir.root': server_path})
     cherrypy.quickstart(root, '/', config=arg.server)
-    cherrypy.log.error('Terminated OK ------------------------', 'APPLICATION')
+    cherrypy.log('Terminated OK ------------------------', 'APPLICATION')
     print('\n- Server terminated OK')
     print('=' * 79)
@@ -68,8 +68,8 @@
 <div class="container">
     <div class="jumbotron drop-shadow">
       <h1>Resultado</h1>
-      <p>Teve <strong>${'{:.1f}'.format(t['grade'])}</strong> valores no teste, numa escala de 0 a 20.</p>
-      <p>Pode desligar.</p>
+      <p>Teve <strong>${'{:.1f}'.format(t['grade'])}</strong> valores na escala de 0 a 20.</p>
+      <p>Pode desligar o computador.</p>
       <!-- <p><a class="btn btn-primary btn-lg" href="#" role="button">Learn more</a></p> -->
     </div>
@@ -107,7 +107,7 @@
                 </tr>
                  % endfor
             </tbody>
-            <tfoot>
+<!--             <tfoot>
                 <tr>
                     <th>Média (não ponderada)</th>
                     <th colspan="2">
@@ -127,7 +127,7 @@
                         </div>
                     </th>
                 </tr>
-            </tfoot>
+            </tfoot> -->
         </table>
     </div> <!-- panel -->
 </div> <!-- container -->
@@ -62,32 +62,25 @@
           <div class="col-xs-4">
             <img src="/logo_horizontal.png" class="img-responsive" />
           </div>
-          <div class="col-xs-8">
-            <h4 class="text-right">
-            <small>
-              $\sqrt{2\pi}$
-            </small>
-            </h4>
-          </div>
-        </div>
-
-        <form method="post" action="/auth/login" class="form-signin">
-        <div class="row">
-          <input type="hidden" name="from_page" value="${from_page}">
-
-          <input type="text" name="uid" class="form-control" placeholder="Número" required autofocus>
+          <div class="col-xs-4">
+          </div>
-          <input type="password" name="pw" class="form-control" placeholder="Password" required>
-        </div>
-        <div class="row">
-          <button class="btn btn-primary" type="submit">
-            <span class="glyphicon glyphicon-log-in"></span>&nbsp; Entrar
-          </button>
+          <div class="col-xs-4">
+          <h4>Identificação:</h4>
+            <form method="post" action="/auth/login" class="form-signin">
+              <input type="hidden" name="from_page" value="${from_page}">
+              <div class="form-group">
+                <input type="text" name="uid" class="form-control" placeholder="Número" required autofocus>
+                <input type="password" name="pw" class="form-control" placeholder="Password" required>
+              </div>
+              <button class="btn btn-primary" type="submit">
+                <span class="glyphicon glyphicon-log-in"></span>&nbsp; Entrar
+              </button>
+            </form>
+          </div>
         </div>
-        </form>
       </div>
-
     </div> <!-- /container -->
   </body>
 </html>
@@ -128,9 +128,8 @@
             % if t['practice'] and 'grade' in t:
                 <div class="jumbotron drop-shadow">
                     <h1>Resultado</h1>
-                    <p>Teve <strong>${'{:.1f}'.format(t['grade'])}</strong> valores no teste.</p>
+                    <p>Teve <strong>${'{:.1f}'.format(t['grade'])}</strong> valores.</p>
                     <p>Se quiser, pode corrigir e submeter o teste novamente.<br>Para terminar escolha a opção 'Sair' no menu.</p>
-
                 </div>
             % endif
@@ -241,9 +240,6 @@
                                     </div>
                                 % else:
                                     <div class="alert alert-danger" role="alert">
-                                        <p>
-                                            ${random.choice(t['offensive']) if 'offensive' in t else ''}
-                                        </p>
                                         <span class="glyphicon glyphicon-remove" aria-hidden="true"></span>
                                         ${round(q['grade'] * q['points'] / total_points * 20.0, 1)} pontos
                                     </div>
@@ -32,6 +32,11 @@ def read_configuration(filename, debug=False, show_points=False, show_hints=Fals
     test['practice'] = bool(test.get('practice', practice))
     test['debug'] = bool(test.get('debug', debug))
     test['show_ref'] = bool(test.get('show_ref', show_ref))
+    test['path'] = str(test.get('path', '.'))  # questions dir  FIXME use os.path
+    # FIXME check that the directory exists
+    if not os.path.exists(test['path']):
+        print('  * Questions path "{0}" does not exist. Fix it in "{1}".'.format(test['path'], filename))
+        sys.exit(1)
     test['save_answers'] = bool(test.get('save_answers', save_answers))
     if test['save_answers']:
         if 'answers_dir' not in test:
@@ -43,13 +48,18 @@ def read_configuration(filename, debug=False, show_points=False, show_hints=Fals
     if 'database' not in test:
         print('  * Missing database in the test configuration.')
         sys.exit(1)
+    if not os.path.exists(test['database']):
+        print('  * Database "{0}" not found.'.format(test['database']))
+        sys.exit(1)
+
     if isinstance(test['files'], str):
         test['files'] = [test['files']]
     # replace ref,points by actual questions from pool
     pool = questions.QuestionsPool()
-    pool.add_from_files(test['files'])
+    pool.add_from_files(files=
+        test['files'], path=test['path'])
     for i, q in enumerate(test['questions']):
         # each question is a list of alternative versions, even if the list
	@@ -1,37 +0,0 @@	@@ -1,37 +0,0 @@
1	-# -- coding: utf-8 --
2	-
3	-[global]
4	-;environment= 'production'
5	-
6	-; number of threads running
7	-server.thread_pool= 10
8	-
9	-; Host address and port
10	-; set socket_port = 443 if SSL is enabled below, 8080 otherwise
11	-; if port is 443 then the server should be run as root. SSL also works on 8080.
12	-server.socket_host = '0.0.0.0'
13	-server.socket_port = 8080
14	-
15	-; Uncomment to enable SSL (see README.txt on how to generate certificates)
16	-; server.ssl_module can be 'builtin' or 'pyopenssl'.
17	-; server.ssl_module = 'builtin'
18	-; server.ssl_certificate = 'certs/webserver.crt'
19	-; server.ssl_private_key = 'certs/webserver.key'
20	-; not required for snakeoil:
21	-; server.ssl_certificate_chain = 'ca_certs.crt'
22	-
23	-log.screen = False
24	-log.error_file = 'logs/errors.log'
25	-log.access_file = 'logs/access.log'
26	-
27	-tools.sessions.on = True
28	-tools.sessions.timeout = 240
29	-tools.sessions.storage_type = 'ram'
30	-tools.sessions.storage_path = 'sessions'
31	-tools.auth.on = True
32	-
33	-[/]
34	-tools.staticdir.root = os.path.normpath(os.path.abspath(os.path.curdir))
35	-tools.staticdir.dir = 'static'
36	-tools.staticdir.on = True
37	-; tools.staticdir.debug = True
	@@ -44,13 +44,13 @@		@@ -44,13 +44,13 @@
44	ref: question-colors	44	ref: question-colors
45	type: textarea	45	type: textarea
46	text: Write names of the three basic colors.	46	text: Write names of the three basic colors.
47	- correct: demo/correct-question.py	47	+ correct: correct-question.py
48	hint: They start by RGB and order does not matter.	48	hint: They start by RGB and order does not matter.
49	# ---------------------------------------------------------------------------	49	# ---------------------------------------------------------------------------
50	-	50	-
51	ref: question-whatever	51	ref: question-whatever
52	type: generator	52	type: generator
53	- script: demo/generate-question.py	53	+ script: generate-question.py
54	arg: "11,120"	54	arg: "11,120"
55	# the script should print a question in yaml format like the ones above.	55	# the script should print a question in yaml format like the ones above.
56	# Print only the dictionary, not the list (hiffen).	56	# Print only the dictionary, not the list (hiffen).
	@@ -1,3 +0,0 @@		@@ -1,3 +0,0 @@
1	-# ignore everything except .gitignore
2	-*
3	-!.gitignore
4	\ No newline at end of file	0	\ No newline at end of file