Aurum Instructor Documentation ============================== This documentation shows instructors how to use Aurum, including creating and registering repository. Get Started ----------- If you are using this tool, we assume you have basic understand of the importance of using modern version control system such as Git, Mercurial and even SVN. Terminology ~~~~~~~~~~~ * exercise repository contains test codes Requirments ~~~~~~~~~~~ To interact with Aurum, you must have the followings ready: 1. install aurum-cli (command-line interface) 2. an editor 3. BitBucket / GitHub account ready As of now, we recommend using **BitBucket** because with an ``.edu`` email address, you can host unlimited **private** repositories there. We do not want to expose our exercise repository (containing test code) to the public! Add teamglass7311 to repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For each exercise repoistory, you must give ``teamglass7311`` the BitBucket user **READ** access. .. figure:: images/user/admin1.png Visit your exercise repo and click on **Admin** .. figure:: images/user/admin2.png Click on **Access Management** .. figure:: images/user/admin3.png Enter **teamglass7311** and click on **READ** button. .. figure:: images/user/admin4.png On success, you should now see **teamglass7311** is given **READ**. Once this is done, ``teamglass7311`` (our dear Aurum service) will be able to clone down your exercise repository during the execution. Create an exercise repository ----------------------------- An exercise repository should contains the following, ideally * config.yaml * runner.py * test_*.py An example written for C programming language is provided `here `_. This example contains more than necessary because of the way ``test_.py`` was written. config.yaml ~~~~~~~~~~~ This file specifies how Aurum should execute and test student's solution. See :ref:`config_yaml` for configuration details. An example (taken from the example repo above):: student_file: student.c build: build_system: make build_file: student.c run: student_soln_exe: student test_runner_file: runner.py nose_report_file: nosetests.xml run_student_file: True run_nosetests: True full_scores: test_binary_exists: full_score: 1 description: Does the student binary exists? test_binary_same: full_score: 1 description: Are the binary produced by student and instructor equal? .. warning:: Avoid using TAB in configuration file. Use 4 spaces instead of tab, otherwise Aurum will fail to read the configuration file! runner.py ~~~~~~~~~ This file should not be changed. We use this to run nosetest on student solution. Simply grab the source code `here `_ we release. .. note:: We will add support for the generating Aurum compatible repository skeleton. Stay tune. test_*.py ~~~~~~~~~~~~~~~ Instructor must write a valid unittest in Python. It can be a test suite comprised of multiple unittests in multiple ``.py`` files. We use `nosetest `_ on the server to run your test codes against student's solution. The example we given above has one test file called ``test_c_exercise.py``.:: import os import unittest from yaml import safe_load from subprocess import Popen, PIPE class TestProblem1(unittest.TestCase): def setUp(self): data_config = '' with open('.datalist.yaml', 'r') as f: data_config = safe_load(f) self.data_location = data_config['dirname'] def tearDown(self): pass def test_binary_exists(self): yes = os.path.exists(os.path.join(os.getcwd(),\ 'student')) self.assertTrue(yes) def test_outputs_hello_world(self): cmd1 = ['./student'] p1 = Popen(cmd1, stdout=PIPE, stderr=PIPE) out1, err1 = p1.communicate() cmd2 = ['./instructor_binary'] p2 = Popen(cmd2, stdout=PIPE, stderr=PIPE) out2, err2 = p2.communicate() self.assertEqual(out1, out2) self.assertEqual(err1, err2) This test code tests whether the output from student is exactly the same as the expected output in the instructor's exectuable. In this case, the instructor's exectuable was packaged into the repository. Make sure you name them wisely so students can't cheat by guessing common names such as ``instructor.exe``. Language specific ----------------- Although :ref:`config_yaml` will explain all the options, it is compelling to repeat some of them here to address language specific configuration. Compiled languages ~~~~~~~~~~~~~~~~~~ If you are running a C or C++ course, you probably want to supply a ``Makefile`` so that Aurum will grab it (according to the build configuration in ``config.yaml``) and build the student's solution properly (with some libraries, e.g). A simple Makefile may look like this :: all: main main: student.c gcc -Wall -o student student.c This rule must match ``student_soln_exe`` configuration option in ``config.yaml``, which specify the name of the executable based on student's solution (which is usually used to test the output like the example above.) For example, we have :: student_file: student.c build: build_system: make build_file: student.c run: student_soln_exe: student test_runner_file: runner.py .. warning:: Make sure you test your Makefile before distributing your repository. Makefile is also picky about spaces and indentication! Python ~~~~~~ Python can be compiled or interperted depending on the type of Python. We currently support CPython only. If there is no explicit compilation required (besides CPython's auto compilation to byte codes for optimization), then you can skip ``build`` option in the configuration. See :ref:`config_yaml` for configuration details.