.. _writing-exercises:

Writing Exercises
#################

An exercise is mainly composed of:

- a title,
- instructions,
- a correction script.

The instructions are written in Markdown_ and correction scripts are
implemented in Python.

Everything is stored in the database so it is editable from the admin
interface, and from the API, there's a nice command line tool to work
from your beloved shell.


Exercise instructions
*********************

That's what users are facing first to resolve an exercise, but you
write it in Markdown_ and it gets rendered to them.

You can be as simple as:

.. code:: markdown

    Write a single line of code displaying the number 42.

And go as explicit as:

.. code:: markdown

    Write a single line of code displaying the number 42.

    ## What you'll learn

    In this exercise you'll learn to *use* a function by trying the
    [`print`](https://docs.python.org/3/library/functions.html#print) builtin
    function.

    A function is a named thing that does something specific:

    - the `print` function displays what's given to it,
    - the `len` function measures the thing we give to it,
    - …

    The syntax to make a function work is called a "function call", to make `print`
    display the string `"Hello world"` the syntax is:

    ```python
    print("Hello world")
    ```

    ## Advice

    You'll need a [number](https://docs.python.org/3/tutorial/introduction.html#numbers)
    and the [`print()`](https://docs.python.org/3/library/functions.html#print) builtin function.

    I'm not asking to print the string "42", (composed of two chars), but the number 42.

    Don't hesitate to hit the "Submit" button, do as many test as you
    like, until you feel confident you understand how `print` works.

This one can be seen here: https://www.hackinscience.org/exercises/print-42


Correction scripts
******************

The correction script is a Python script, it gets executed in a
sandboxed environment along with the student answer.

In the sandboxed environment your correction script is executed as
``python check.py`` and the student answer is placed in the same
directory in a file named ``solution``.

The protocol is:

- To indicate the solution is wrong: print an error message in
  Markdown and exit with nonzero.
- To indicate the solution is right: print a congratulation message in
  Markdown and exit with zero.

If the congratulation message is omitted, a default one will be generated.


Example in Pure Python
======================

A very short correction script may look like this:

.. code:: python

  from pathlib import Path

  if "My name is: " not in Path('solution').read_text():
      print("That's wrong.")
      exit(1)


It just checks that the student wrote "My name is: ..." in the file
(not that the answer is a Python script that prints, no, just that the
user wrote the text in its answer).


Example with a correction library
=================================

Writing a lot of correction scripts is repetitive, so I wrote a
library: `correction-helper`_ to help me.

Here's a more realistic example of a short correction script for a
Python exercise:

.. code:: python

  from pathlib import Path

  import correction_helper as checker

  checker.exclude_file_from_traceback(__file__)
  Path("solution").rename("solution.py")

  output = checker.run("solution.py")
  if "hello world" not in output.lower():
      checker.fail(
          'You should print "Hello world!", not something else to validate this exercise.',
          "You printed:",
          checker.code(output)
      )


Here:

- ``checker.run("solution.py")`` is a shortcut to run the answer as a
  Python file and get its output.
- ``checker.fail(...)`` is a shortcut to ``print`` and ``exit(1)``
- ``checker.code(...)`` formats a string as a Markdown literal block
  so it gets rendered nicely to the user.

The correction-helper library also catches exceptions from user code
and display them using `friendly`_.


Testing user functions
======================

Running the whole script once is OK for easy exercises, but soon
you'll want to check user functions with different arguments, it's a
bit like writing unit tests against student code.

`correction-helper`_ provides a tool to run student code, catching
exceptions, and passing them to `friendly`_ for nice display, so we
don't care about exceptions when writing checks.

You could call the student function directly and handle exceptions
yourself, it would work.

To call student function just import their code first:

.. code:: python

  from pathlib import Path

  Path("solution").rename("solution.py")

  import solution


But already it can raise an exception (think: ``SyntaxError``), so
better handle the case:

.. code:: python

  from pathlib import Path
  import correction_helper as checker

  checker.exclude_file_from_traceback(__file__)

  Path("solution").rename("solution.py")

  with checker.student_code():
      import solution

That's better!! Now we can call functions from the student answer:

.. code:: python

  with checker.student_code():
      result = solution.circle_perimeter(0)

  if result != 0:
      checker.fail("Blah blah describe problem to student blah blah")

Try to put the **minimum** inside the ``student_code`` manager: you
don't want errors in your code to be reported as errors in the student
code!


Hints
=====

Start simple.

Don't guess what could go wrong. Then do the exercise yourself and
iteratively improve on the messages while you're facing situations
where the correction script could have done better. Once you're happy
give the exercises to friends as a test, and iterate again to enhance
the messages.

All hackinscience.org exercises are open-source, so don't hesitate to
take a look at how we write our correction bots:

https://framagit.org/hackinscience/hkis-exercises


Checking other languages
========================

As the correction script is a Python file, so we're fully free to test
anything in any way.

As an example it could take the user answer, compile it with GCC, and
test the produced binary:

.. code:: python

  import sys
  from subprocess import run, PIPE
  from textwrap import indent
  from pathlib import Path

  Path("solution").rename("solution.c")

  def literal(text):
      """Format the given text as a Markdown literal block."""
      return "    :::text\n" + indent(str(text), "    ")

  gcc = run(
      ["gcc", "solution.c", "-o", "solution"], stderr=PIPE, stdout=PIPE, encoding="UTF-8"
  )
  if gcc.stderr:
      print("Failed compiling your code:")
      print(literal(gcc.stderr))
      sys.exit(1)

  output = run(["./solution"], stdout=PIPE, stderr=PIPE, encoding="UTF-8")

  if output.stdout == "Hello World\n":
      print("Congratulations!!")
      sys.exit(0)

  if output.stdout == "Hello World":
      print("It would be better with a newline at the end, written as `\\n` in C.")
      sys.exit(1)

  message = ["Naupe. Your code printed on `stdout`:", literal(output.stdout)]
  if output.stderr:
      message.extend(["And on `stderr`:", literal(output.stderr)])
  message.append("While I expected it to print: `Hello World\\n`")
  print(*message, sep="\n\n")
  sys.exit(1)


Creating a new exercise
***********************

It could be all done from the admin, but let's explore a better way.

It's a good practice to synchronize your exercises with a git
repository to keep an history of your modifications, to write down why
you're changing things, as another backup of your work, and to test
locally.

To do so we'll use a command-line tool, install it first::

    pip install genepy-cli

You'll need to login with the tool::

    genepy login

You can check if you're correctly logged by using::

    genepy profile

Once you're ready, create your first exercise::

    genepy new python/is-anagram

The argument is both the directory created on your disk to store the
exercise files, and the URL your exercise will have on the website, it
should now looks like:

.. code::

  python/is-anagram/
  ├── check.py
  ├── initial_solution
  ├── meta
  ├── wording_en.md
  └── wording_fr.md


- ``meta`` just contains metadata, you can adjust title, position, and is_published here.
- ``wording_*.md`` contains the instructions, translated in english and french.
- ``check.py`` is the correction script.

Edit the files, see previous sections in this page if you need
guidance to redact those parts.

The correction script is not easy to write feel free to get
inspiration by reading how I do it for my exercises at
https://framagit.org/hackinscience/hkis-exercises.

Once you're ready to test, you can upload your exercise::

    genepy push python/is-anagram

When I'm happy with the files, I push them using::

  genepy push exercises/dyck-words/

(or just ``genepy push`` to push them all!)

After a few tests, after having friends test it, and so on, it's time
to make it public by changing ``is_published`` to ``true`` in the
``meta`` file.

.. _Markdown: https://daringfireball.net/projects/markdown/
.. _correction-helper: https://pypi.org/project/correction-helper/
.. _friendly: https://pypi.org/project/friendly/
.. _hkis-website: https://framagit.org/hackinscience/hkis-website/
.. _hkis-exercises: https://framagit.org/hackinscience/hkis-exercises/