.. _config_yaml:


Exercise Configuration
======================

In this section, we will detailed all the configuration options
usable in setting up an exercise repository writting ``config.yaml``.

Required Configuration
----------------------

The followings must be provided to drive Aurum to run students' codes.

.. code-block:: python
    
    student_file: <filename>
    run:
        student_soln_exe: <filename>
        test_runner_file: runner.py
    
    nose_report_file: nosetests.xml
    run_student_file: True
    run_nosetests: True
    
    full_scores:
        test_<function_case_one_name>:
            full_score: <int>
            description: <string>

``<  >`` specify variables and others are default values. For example, ``run_student_file:`` should
be ``True`` for most instructors. 


.. option:: student_file

    The name of the file in which the student code will be written into. **main.py**, **main.c**
    are recommended for the ease:

.. option:: run

    Specify the run procedure parameters.

    **student_soln_exe**
        The name of student's executable. For compiled language, it is usually the name of the exectuable.
        For Python, it is usually the same as ``student_file`` above.

    **test_runner_file**
        The name of the script running nosetests. We recommend grabbing our ``runner.py`` `here <https://gist.github.com/3801860>`_

    **nose_report_file**
        The name of the nosetests report. Default we ask to use ``nosetests.xml`` so please don't change it unless there is an usecase.

    **run_student_file**
        Defaults to ``True`` for most usecase.

    **run_nosetests**
        Defaults to ``True`` for most usecase.

.. option:: full_score

    Specify the scoring metrics.

    **test_<function_case_N_name>**
        Each unittest testcase (individual unittest function) should be reflected here.
        If the test case is called ``def test_hello_world_is_seen``, then instructor
        should put ``test_hello_world_is_seen`` into the configuration file.

        For each test, please give an integer for **full_score** to indicate the scale. 
        If there are four cases to test, and each case worth 1 point, a student scores
        3 out of 4 (passed 3 tests) will receive 75% on the assignment.

        For each test, please give a brief one-line description on what the test does.


Build Configuration
-------------------

These options are necessary for compiled languages such as C and C++. Please note if a solution requires
compilation, please specify it.

.. code-block:: yaml

    build:
        build_system: <name of the make system>
        build_file: <name of the file to be built>

    run:
        student_soln_exe: <executable name>
        test_runner_file: runner.py

.. option:: build

    Specify the build procdeure in Aurum for solutions that required compilation.

    **build_system**
        Only ``make`` is supported. So please fill in ``make``.

    **build_file**
        The main file to be built. (e.g. main.c written by student - in the case (usually),
        building the student's solution and tests the output)

.. option:: run
   
   Specify the run procedure in accordance to the build procedure.

   **student_soln_exe**
       As required by the basic configuration above, if the exercise requires compilation,
       then this option should reflect the name of the executable. 

       For example, in the ``Makefile`` we specify the name of the output file, and usually
       we can use that name in this field.

   **test_runner_file**
       As required by the basic configuration above, this is necessary and we recommend no
       change. Grab the official ``runner.py`` from `here <https://gist.github.com/3801860>`_.


Output Artifacts Configuration
------------------------------

By that, we mean images or files generated when running a script.

.. code-block:: yaml

   show_outfiles: ["<file_extension1>", "<file_extension2>", ...]


.. option:: show_outfiles

   Specify what needs to be shown to the users after running the code.
   Notably, we support image outputs. For example, if we want to display a set of
   PNG images a student generated, we can specify the rule as ``show_outfiles: ["*.png"]``


Example for C
-------------

.. code-block:: yaml

    student_file: student.c
    build:
        build_system: make
        build_file: student.c
    run:
        student_soln_exe: student
        test_runner_file: runner.py

    nose_report_file: nosetests.xml
    run_student_file: True
    run_nosetests: True

    full_scores:
        test_binary_exists:
            full_score: 1
            description: Does the student binary exists?
        test_binary_same:
            full_score: 1
            description: Are the binary produced by student and instructor equal?


Example for Python
------------------

.. code-block:: yaml

    student_file: student.py
    run:
        student_soln_exe: student.py
        test_runner_file: runner.py
    
    nose_report_file: nosetests.xml
    run_student_file: True
    
   full_scores:
       test_it_runs:
           full_score: 1
           description: Does it run without crashing?
       test_image_found:
           full_score: 1
           description: Did you save out an image?
       test_image_matches:
           full_score: 2
           description: Is the image the correct image with colorbar?