Exercise Configuration

In this section, we will detailed all the configuration options usable in setting up an exercise repository writting config.yaml.

Required Configuration

The followings must be provided to drive Aurum to run students’ codes.

student_file: <filename>
run:
    student_soln_exe: <filename>
    test_runner_file: runner.py

nose_report_file: nosetests.xml
run_student_file: True
run_nosetests: True

full_scores:
    test_<function_case_one_name>:
        full_score: <int>
        description: <string>

<  > specify variables and others are default values. For example, run_student_file: should be True for most instructors.

student_file

The name of the file in which the student code will be written into. main.py, main.c are recommended for the ease:

run

Specify the run procedure parameters.

student_soln_exe

The name of student’s executable. For compiled language, it is usually the name of the exectuable. For Python, it is usually the same as student_file above.

test_runner_file

The name of the script running nosetests. We recommend grabbing our runner.py here

nose_report_file

The name of the nosetests report. Default we ask to use nosetests.xml so please don’t change it unless there is an usecase.

run_student_file

Defaults to True for most usecase.

run_nosetests

Defaults to True for most usecase.

full_score

Specify the scoring metrics.

test_<function_case_N_name>

Each unittest testcase (individual unittest function) should be reflected here. If the test case is called def test_hello_world_is_seen, then instructor should put test_hello_world_is_seen into the configuration file.

For each test, please give an integer for full_score to indicate the scale. If there are four cases to test, and each case worth 1 point, a student scores 3 out of 4 (passed 3 tests) will receive 75% on the assignment.

For each test, please give a brief one-line description on what the test does.

Build Configuration

These options are necessary for compiled languages such as C and C++. Please note if a solution requires compilation, please specify it.

build:
    build_system: <name of the make system>
    build_file: <name of the file to be built>

run:
    student_soln_exe: <executable name>
    test_runner_file: runner.py

build

Specify the build procdeure in Aurum for solutions that required compilation.

build_system

Only make is supported. So please fill in make.

build_file

The main file to be built. (e.g. main.c written by student - in the case (usually), building the student’s solution and tests the output)

run

Specify the run procedure in accordance to the build procedure.

student_soln_exe

As required by the basic configuration above, if the exercise requires compilation, then this option should reflect the name of the executable.

For example, in the Makefile we specify the name of the output file, and usually we can use that name in this field.

test_runner_file

As required by the basic configuration above, this is necessary and we recommend no change. Grab the official runner.py from here.

Output Artifacts Configuration

By that, we mean images or files generated when running a script.

show_outfiles: ["<file_extension1>", "<file_extension2>", ...]

show_outfiles

Specify what needs to be shown to the users after running the code. Notably, we support image outputs. For example, if we want to display a set of PNG images a student generated, we can specify the rule as show_outfiles: ["*.png"]

Example for C

student_file: student.c
build:
    build_system: make
    build_file: student.c
run:
    student_soln_exe: student
    test_runner_file: runner.py

nose_report_file: nosetests.xml
run_student_file: True
run_nosetests: True

full_scores:
    test_binary_exists:
        full_score: 1
        description: Does the student binary exists?
    test_binary_same:
        full_score: 1
        description: Are the binary produced by student and instructor equal?

Example for Python

 student_file: student.py
 run:
     student_soln_exe: student.py
     test_runner_file: runner.py

 nose_report_file: nosetests.xml
 run_student_file: True

full_scores:
    test_it_runs:
        full_score: 1
        description: Does it run without crashing?
    test_image_found:
        full_score: 1
        description: Did you save out an image?
    test_image_matches:
        full_score: 2
        description: Is the image the correct image with colorbar?