From 0057db016de7e158802129f03427afbb711ef777 Mon Sep 17 00:00:00 2001 From: Miguel Barão Date: Tue, 1 Nov 2016 18:48:23 +0000 Subject: [PATCH] - documentation update --- BUGS.md | 5 +++-- MANUAL.md | 392 +++++++++++++++++++------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- serve.py | 2 +- 3 files changed, 23 insertions(+), 376 deletions(-) diff --git a/BUGS.md b/BUGS.md index 2c8ccb3..43f8d3d 100644 --- a/BUGS.md +++ b/BUGS.md @@ -2,11 +2,12 @@ # BUGS - se directorio logs não existir no directorio actual (não perguntations) rebenta. -- usar thread.Lock para aceder a variaveis de estado? -- servidor nao esta a lidar com eventos scroll/resize # TODO +- SSL +- usar thread.Lock para aceder a variaveis de estado? +- servidor nao esta a lidar com eventos scroll/resize - se ocorrer um erro na correcçao avisar aluno para contactar o professor. - implementar practice mode. - usar http://wtfforms.com para radio e checkboxes diff --git a/MANUAL.md b/MANUAL.md index 3844c2b..793062a 100644 --- a/MANUAL.md +++ b/MANUAL.md @@ -6,14 +6,12 @@ After installing python and the required packages as described in the `README.md ``` $ cd PATH/TO/perguntations -$ ./initdb.py +$ ./initdb.py --demo New database created: students.db 10 users inserted: 0 - Professor (administrator) 1465 - Gil Vicente - . - . - . + ... - ... 1924 - Alexandre O'Neill $ mv students.db demo/ $ ./serve.py demo/test.yaml @@ -28,16 +26,15 @@ $ ./serve.py demo/test.yaml You can now open a browser and point it to the server address, e.g. `http://127.0.0.1:8080`. -Enter `0` for the number and any password. This will be your password from now on. +Enter `0` for the number and choose any password. This will be your password from now on. After login, you will see an administrator page. This page allows you to: -- Generate a sample test just for you to see if it looks right. - Allow students to login to the test. This can be done one at a time using the checkboxes on the left of each student, or allow all students by clicking the button in the bottom of the page. - Review a test done by a student by clicking the grade colorbar on the right. If a student has multiple tests, then several bars will be shown. A tooltip shows the date/time for each test. - Reset student passwords. Students that already have a password will have a label `PW` next to the student name. To reset the password, go to the bottom of the page and insert the student number in the box and press the button to reset the password. The label `PW` will disappear. -- Online students will be marked and the table on the top of the page will show some more information (login time, ip address, etc) -- If a student moves away from the browser (e.g. for cheating), a red bar appears along with the status `unfocus`. Screen saver is also reported as unfocus, so some care should be taken... +- Online students will be highlighted and the table on the top of the page will show some more information (login time, ip address, etc) +- If a student moves unfocus the browser window on his computer (e.g. for cheating), a red bar appears along with the status `unfocus`. Screen saver is also reported as unfocus, so some care should be taken... - It is also possible to add new students by clicking the button on the bottom of the page. ## Answering a test @@ -50,380 +47,29 @@ After answering all the questions, the test is submited by clicking the button o ## Creating a new test -A test is just a yaml file with the configuration for that test. Look at `demo/test.yaml` for an example. +A test is specified in a single yaml file. Look at `demo/test.yaml` for an example and documentation. ## Creating questions -Questions are defined in yaml files. Each yaml file contains a list of questions +Questions are defined in yaml files. Each yaml file contains a list of questions from which some can be chosen to be part of a test. +The types of questions supported are: +- `radio` questions are multiple choice where only one can be selected. +- `checkbox` questions are also multiple choice but several can be selected. +- `text` and `text_regex` questions provide a line to write the answer. +- `textarea` provide a multiline text box to write the answer (e.g. for computer code). The answer is corrected by calling an external program that is not part of the server and is up to you to right it. +Besides these, there are `information` and `alert` types which just show text but do not expect answers. +All the types above are predifined questions. It is also possible to generate questions by code. These are defined as type `generator` and an external program must be provided to generate a question of the types above. +An example file with these types of questions is provided in `demo/questions/questions.yaml` along with accompanying files under the directory `demo/questions`. +## Testing a new test +Before using a test in product you should test is very carefully. After writing the questions and the test configuration, my recommendation is to: - - - - - - - -Before using the program you need to - -1. Edit `config/server.conf` in the server directory and define - - Logging - - `log.error_file= '/Users/USERNAME/Library/Logs/Perguntations/errors.log'` - - `log.access_file= '/Users/USERNAME/Library/Logs/Perguntations/access.log'` - - You must create the directories if they do not exist already. - - Setting these locations to empty strings `''` disables logging. - - - Sessions - If `tools.sessions.storage_type='file'` sessions are saved on the file system in the location given in `tools.sessions.storage_path`. Restarting the server will maintain the sessions active. - - If `storage_type='ram'` (default) no files are stored but restaring the server will reset sessions. - - You should give enough time in the `tools.sessions.timeout` to complete an exam. The default is 240 minutes (4 hours). - -1. Create the students database (see below) -1. Create questions (see below) -1. Create a test (see below) - -### Create students database - -We need a sqlite3 database to store students, passwords, test results, and questions results, etc. - -The database can be initialized from a list of students in CSV format by running in the terminal - -```sh -./initdb_from_csv.py list_of_students.csv -``` - -This script will create a new sqlite3 database with the correct tables and insert the students with empty passwords. -It also adds a special user number 0. This is the administrator user (Professor). - -The passwords will be defined on the first login. - -### Create new questions - -Questions are defined in `yaml` files and can reside anywhere in the filesystem. -Each file contains a list of questions, where each question is a dictionary. Example - -```yaml -- - ref: question-1 - type: radio - text: Select the correct option - options: - - correct - - wrong - -- - ref: question-2 - type: checkbox - text: Which ones are correct? - options: - - correct - - correct - - wrong - correct: [1, 1, -1] - hint: There are two correct answers! -``` - -There are several kinds of questions: - -- __information__: nothing to answer -- __radio__: only one option is correct -- __checkbox__: several options are correct -- __text__: compares text with a list of accepted answers -- __text_regex__: matches text agains regular expression -- __textarea__: send text to an external script for validation -- __generator__: the question is generated from an external script, the actual question generated can be any of the above types. - -Detailed information on each question type is described later on. - - -### Creating a new test - -A test is a file in `yaml` format that can reside anywhere on the filesystem. It has the following structure: - -```yaml -ref: this-is-a-key -title: Titulo do teste -database: db/mystudents.db - -# Will save the entire test of each student in JSON format. -# If tests are to be saved, we must specify the directory. -# The directory is created if it doesn't exist already. -# The name of the JSON files will include the student number, test -# reference key, date and time. -save_answers: True -answers_dir: ans/asc1_test4 - -# Some questions can contain hints, embedded videos, etc -show_hints: True - -# Each question has some number of points. Show them normalized to 0-20. -show_points: True - -# In train mode, the correction of the test is shown and the test can -# be repeated -practice_mode: True - -# Show the data structures obtained from the test and the questions -debug: False - -# Show the file and ref field of each question -show_ref: True - -# ---------------------------------------------------------------------------- -# Location of the questions files (absolute path or relative to current dir) -path: questions - -# This are the questions files to be imported. -files: - - file1.yaml - - file2.yaml - - file3.yaml - -# ---------------------------------------------------------------------------- -# This is the actual test configuration. Selection of questions and points -# It'a defined as a list of questions. Each question can be a single -# question key or a list of keys from which one is chosen at random. -# Each question has a default value of 1.0 point, but it can be overridden. -# The points defined here do not need to be normalized (it's automatic). -questions: - - ref: - - first-question-1 # randomly choose one from these 3 questions - - first-question-2 - - first-question-3 - points: 0.5 - - - ref: second-question # one question, 1.0 point (unnormalized) - - - third-question # "ref:" not needed in simple cases - -This following one is wrong: - - - wrong-question # missing "ref:" key - points: 2 -``` - -Some of the options have default values if they are omitted. The defaults are the following: - -```yaml -ref: filename.yaml -title: '' -save_answers: False -show_hints: False -show_points: False -practice_mode: False -show_ref: False -debug: False -points: 1.0 -``` -### Running an existing test - -A test is a file in `yaml` format. Just run `serve.py` with the test to run as argument: - -```sh -$ ./serve.py tests_dir/mytest.yaml -``` - -Some defaults can be overriden with command line options. Example - -```sh -$ ./serve.py mytest.yaml --debug --show_points --show_hints --practice_mode --save_answers -``` -To terminate the test just do `^C` on the keyboard. - -## Questions - -Every question should have a `ref` and a `type`. The other keys depend on the type of question. - -### Information - -Not a real question. Just text to be shown without expecting an answer. - -```yaml -- - ref: some-key - type: information - text: Tomorrow will rain. -``` -Correcting an information will always be considered correct, but the grade will be zero because it has 0.0 points by default. - -### Radio - -Only one option is correct. - -```yaml -- - ref: some-key - type: radio - text: The horse is white. # optional (default: '') - options: - - The horse is white - - The horse is not black - - The horse is black - correct: 0 # optional (default: 0). Index is 0-based. - shuffle: True # optional (default: True) - discount: True # optional (default: True) -``` -The `correct` value can also be defined as a list of degrees of correctness between 0 (wrong) and 1 (correct), e.g. if answering "the horse is not black" should be considered half-right, then we should use `correct: [1, 0.5, 0]`. - -Wrong answers discount by default. If there are half-right answers, the discount values are calculated automatically. `discount: False` disables the discount calculation and the values are the ones defined in `correct`. - -### Checkbox - -There can be several options correct. Each option is like answering an independent question. - -```yaml -- - ref: some-key - type: checkbox - text: The horse is white. # optional (default: '') - options: - - The horse is white - - The horse is not black - - The horse is black - correct: [1,1,-1] # optional (default: [0,0,0]). - shuffle: True # optional (default: True) - discount: True # optional (default: True) -``` -Wrong answers discount by default. The discount values are calculated automatically and are simply the symmetric of the correct value. -E.g. consider `correct: [1, 0.5, -1]`, then - -- if the first option is marked the value is 1, otherwise if it's unmarked the value is -1. -- if the second option is marked the value is 0.5, otherwise if it's unmarked the value is -0.5. -- if the third option is marked the value is -1, otherwise if it's unmarked the value is 1. (the student shouldn't have marked this one) - -`discount: False` disables the discount and the values are the ones defined in `correct` if the answer is right, or 0.0 if wrong. - -### Text - -The answer is a line of text. -The server will check if the answer exactly matches the correct one. - -```yaml -- - ref: some-key - type: text - text: What's your favorite color? # optional (default: '') - correct: white -``` -alternatively, we can give a list of acceptable answers -```yaml - correct: ['white', 'blue', 'red'] -``` -### Regular expression - -The answer is a line of text. -The server will check if the answer matches a regular expression. - -```yaml -- - ref: some-key - type: text_regex - text: What's your favorite color? # optional (default: '') - correct: '[Ww]hite' -``` - -Careful: yaml does not support raw text. Some characters have to be escaped. - -### Text area - -The answer is given in a textarea. The text (usually code) is sent to an external program running on a separate process for validation. -The external program should accept input from stdin, and print to stdout a single number in the interval 0.0 to 1.0 indicating the level of correctness. -The server will try to convert the printed message to a float, a failure will give 0.0. - -```yaml -- - ref: some-key - type: textarea - text: write an expression to add x and y. # optional (default: '') - correct: myscript - # optional - lines: 15 -``` - -The script location is the same as the questions file. -An example of a script in python that validades an answer is - -```python -#!/usr/bin/env python3.4 - -import sys -s = sys.stdin.read() -if s == 'Alibaba': - print(1.0) -else: - print(0.0) -exit(0) -``` - -but any script language or executable program can be used for this purpose. - - -### Generator - -A generator question will run an external program that is expected to print a question in yaml format to stdout. After running the generator, the question can be any of the other types (but not another generator!). - -```yaml -- - ref: some-key - type: generator - script: path/to/generator_script - # arg: "optional string passed on to stdin of the script" -``` - -An example of a question generator is the following - -```python -#!/usr/bin/env python3.4 -from random import randint -import sys - -# read arguments from stdin and convert to integers -arg = sys.stdin.read() -a,b = (int(n) for n in arg.split(',')) - -# generate question -x = randint(a, b) -y = randint(a, b) -s = ''' -ref: addition -type: text -text: How much is {0} plus {1}? -correct: {2} -'''.format(x, y, x + y) - -# send question to stdout -print(s) -``` - -## Writing good looking questions - -The text of the questions (and options in radio and checkbox type questios) is parsed as markdown and code is prettyfied using Pygments. Equations can be inserted like in LaTeX and are rendered using MathJax. - -A good way to define multiple lines of text in the questions is to use the bar |. Yaml will use all the text that is indented to the right of that column. Example - -```yaml - text: | - Text is parsed as __markdown__. We can include equations $\sqrt{\pi}$ like in LaTeX - and pretty code in several languages - - ```.C - int main(){ - return 0; - } - ``` - # this line stops the text because it is not indented -``` - +1. `./serve.py --allow-all shiny_new_test.yaml` where the `--allow-all` option avoids having to use a second browser for the admin page to explicitly allow a student. +2. If bugs are found, try commenting questions in the `shiny_new_test.yaml` to isolate the bug. Use the `--debug` option to add more information in the webpage. diff --git a/serve.py b/serve.py index 838d410..4f9965f 100755 --- a/serve.py +++ b/serve.py @@ -121,7 +121,7 @@ class StudentWebService(object): @cherrypy.tools.accept(media='application/json') # FIXME def POST(self, **args): - uid = cherrypy.session.get(SESSION_KEY) + # uid = cherrypy.session.get(SESSION_KEY) if args['cmd'] == 'focus': v = json.loads(args['value']) self.app.set_student_focus(uid=args['number'], value=v) -- libgit2 0.21.2