Writing Exercises#

An exercise is mainly composed of:

  • a title,

  • instructions,

  • a correction script.

The instructions are written in Markdown and correction scripts are implemented in Python.

Everything is stored in the database so it is editable from the admin interface, and from the API.

Exercise instructions#

That’s what users are facing first to resolve an exercise, but you write it in Markdown and it gets rendered to them.

You can be as simple as:

Write a single line of code displaying the number 42.

And go as explicit as:

Write a single line of code displaying the number 42.

## What you'll learn

In this exercise you'll learn to *use* a function by trying the
[`print`](https://docs.python.org/3/library/functions.html#print) builtin
function.

A function is a named thing that does something specific:

- the `print` function displays what's given to it,
- the `len` function measures the thing we give to it,
- …

The syntax to make a function work is called a "function call", to make `print`
display the string `"Hello world"` the syntax is:

```python
print("Hello world")
```

## Advice

You'll need a [number](https://docs.python.org/3/tutorial/introduction.html#numbers)
and the [`print()`](https://docs.python.org/3/library/functions.html#print) builtin function.

I'm not asking to print the string "42", (composed of two chars), but the number 42.

Don't hesitate to hit the "Submit" button, do as many test as you
like, until you feel confident you understand how `print` works.

This one can be seen here: https://www.hackinscience.org/exercises/print-42

Correction scripts#

The correction script is a Python script, it gets executed in a sandboxed environment along with the student answer.

In the sandboxed environment your correction script is executed as python check.py and the student answer is placed in the same directory in a file named solution.

The protocol is:

  • To indicate the solution is wrong: print an error message in Markdown and exit with nonzero.

  • To indicate the solution is right: print a congratulation message in Markdown and exit with zero.

If the congratulation message is omitted, a default one will be generated.

Example in Pure Python#

A very short correction script may look like this:

from pathlib import Path

if "My name is: " not in Path('solution').read_text():
    print("That's wrong.")
    exit(1)

It just checks that the student wrote “My name is: …” in the file (not that the answer is a Python script that prints, no, just that the user wrote the text in its answer).

Example with a correction library#

Writing a lot of correction scripts is repetitive, so I wrote a library: correction-helper to help me.

Here’s a more realistic example of a short correction script for a Python exercise:

from pathlib import Path

import correction_helper as checker

checker.exclude_file_from_traceback(__file__)
Path("solution").rename("solution.py")

output = checker.run("solution.py")
if "hello world" not in output.lower():
    checker.fail(
        'You should print "Hello world!", not something else to validate this exercise.',
        "You printed:",
        checker.code(output)
    )

Here:

  • checker.run("solution.py") is a shortcut to run the answer as a Python file and get its output.

  • checker.fail(...) is a shortcut to print and exit(1)

  • checker.code(...) formats a string as a Markdown literal block so it gets rendered nicely to the user.

The correction-helper library also catches exceptions from user code and display them using friendly.

Testing user functions#

Running the whole script once is OK for easy exercises, but soon you’ll want to check user functions with different arguments, it’s a bit like writing unit tests against student code.

correction-helper provides a tool to run student code, catching exceptions, and passing them to friendly for nice display, so we don’t care about exceptions when writing checks.

You could call the student function directly and handle exceptions yourself, it would work.

To call student function just import their code first:

from pathlib import Path

Path("solution").rename("solution.py")

import solution

But already it can raise an exception (think: SyntaxError), so better handle the case:

from pathlib import Path
import correction_helper as checker

checker.exclude_file_from_traceback(__file__)

Path("solution").rename("solution.py")

with checker.student_code():
    import solution

That’s better!! Now we can call functions from the student answer:

with checker.student_code():
    result = solution.circle_perimeter(0)

if result != 0:
    checker.fail("Blah blah describe problem to student blah blah")

Try to put the minimum inside the student_code manager: you don’t want errors in your code to be reported as errors in the student code!

Hints#

Start simple.

Don’t guess what could go wrong. Then do the exercise yourself and iteratively improve on the messages while you’re facing situations where the correction script could have done better. Once you’re happy give the exercises to friends as a test, and iterate again to enhance the messages.

All hackinscience.org exercises are open-source, so don’t hesitate to take a look at how we write our correction bots:

https://framagit.org/hackinscience/hkis-exercises

Checking other languages#

As the correction script is a Python file, so we’re fully free to test anything in any way.

As an example it could take the user answer, compile it with GCC, and test the produced binary:

import sys
from subprocess import run, PIPE
from textwrap import indent
from pathlib import Path

Path("solution").rename("solution.c")

def literal(text):
    """Format the given text as a Markdown literal block."""
    return "    :::text\n" + indent(str(text), "    ")

gcc = run(
    ["gcc", "solution.c", "-o", "solution"], stderr=PIPE, stdout=PIPE, encoding="UTF-8"
)
if gcc.stderr:
    print("Failed compiling your code:")
    print(literal(gcc.stderr))
    sys.exit(1)

output = run(["./solution"], stdout=PIPE, stderr=PIPE, encoding="UTF-8")

if output.stdout == "Hello World\n":
    print("Congratulations!!")
    sys.exit(0)

if output.stdout == "Hello World":
    print("It would be better with a newline at the end, written as `\\n` in C.")
    sys.exit(1)

message = ["Naupe. Your code printed on `stdout`:", literal(output.stdout)]
if output.stderr:
    message.extend(["And on `stderr`:", literal(output.stderr)])
message.append("While I expected it to print: `Hello World\\n`")
print(*message, sep="\n\n")
sys.exit(1)

Creating a new exercise#

It could be all done from the admin, but let’s explore a better way.

It’s a good practice to synchronize your exercises with a git repository to keep an history of your modifications, to write down why you’re changing things, as another backup of your work, and to test locally.

To do so we’ll use a command-line tool, install it first:

pip install genepy-cli

Here how I work:

First I create my exercises from the admin interface, I give it:

  • a title and its translation,

  • an author,

  • a page (where to display it),

  • optionally a category (a group of exercises in a page).

But don’t mark it as published, you don’t want users to see it.

At this point I don’t write a wording and I don’t write a correction bot, I just save the exercise as is.

Then from a terminal, in a directory dedicated to work on exercises, do:

genepy pull

(You may have to use genepy login first to authenticate.)

It will create a directory for your exercise, it should look like this:

exercises/dyck-words/
├── check.py
├── initial_solution.py
├── meta
├── wording_en.md
└── wording_fr.md

Now I just edit the files locally:

  • meta just contains metadata, I almost never have to touch it.

  • wording_*.md contains the instructions, translated in english and french.

  • check.py is the correction script.

The correction script is not easy to write feel free to get inspiration by reading how I do it for my exercises at https://framagit.org/hackinscience/hkis-exercises.

When I’m happy with the files, I push them using:

genepy push exercises/dyck-words/

(or just genepy push to push them all!)

After a few tests, after having friends test it, and so on, it’s time to make it public by using the admin interface (it can’t be done from the API).

It’s probably a good time to commit and maybe push the exercise, too.