Assertions, tests, and counterexamples.

A test checks a belief about a function, so write the belief down first. The belief is one sentence that says what the function returns, and for which inputs. Then write the tests. A test is a call with a chosen input, plus the value that the belief says the call should return, called the expected value. Work out each expected value from the belief, by hand, before you run the code. The test then compares two things made separately: what the code returns and what the belief says.

For each game, the app stores a score: the team’s points minus the other team’s points. So -1 means the team lost by one point, 0 means a draw, and 2 means the team won by two points. The belief: lowest returns the smallest score in a list that has at least one score. The tests use a small function, check name got want, where got is the value lowest returned and want is the expected value. check returns the line to print: pass when got equals want, and FAIL with both values when they differ. putStrLn prints that line. The full code of check is in the experiment of the next section.

foldr min 0 scores combines the scores with min, starting from 0: for [1, -1, 2] it computes min 1 (min (-1) (min 2 0)). A negative number as an argument needs parentheses, as in (-1).

Write down your belief and each expected value before you run the code.

Passing tests show that the belief holds for their inputs. They say nothing about other inputs. To learn more, look for a counterexample: one input for which the belief is false. One counterexample is enough to show that a belief is wrong, however many tests pass.

Look first at inputs the code was not designed around. Every list in the tests has a game the team lost or drew. A strong team can win every game.

Here lowest [2, 1] returns 0, a draw the team never played, so [2, 1] is a counterexample. foldr min 0 [2, 1] is min 2 (min 1 0), which is 0. Keep the counterexample as a test. After the repair it passes, and it stays as a regression test, the kind of check that Minimal reproductions ends with.

Passing tests show that the belief holds for their inputs; one counterexample shows that it is false.

Counterexamples are most often found at boundaries: inputs at the edge of what the code handles. Try these first: an empty input; a single element; equal values; the answer in the first place and in the last; zero; negative numbers, and no negative numbers at all. A loop that starts or stops one place too early, or a starting value such as 0, is wrong at a boundary and right everywhere else.

For each boundary, work out the expected value from the belief before you run it. The belief may say nothing about an input. This belief says nothing about an empty list. That is a finding too: decide what should happen, and add the decision to the belief.

The repair uses foldr1 min, which combines the scores with min and takes its starting value from the list itself. minimum from the Prelude does the same job and is the usual way to write it. Both need at least one score, and both stop the program on an empty list. The table in this section runs the repaired lowest on one input for each boundary.

Try the edges first: empty, one, equal, first and last, zero, and negative.

A test that cannot fail still prints pass. It tells you nothing. Three kinds are common.

The first kind has no comparison. It calls the function and prints the result. Nothing in it can fail.

The second kind gets its expected value from the code under test. It calls the same function again, or it repeats the same formula. So it compares the code with itself.

The third kind has an expected value copied from the output of one run. It checks that the output does not change. If that output was wrong, the test keeps the bug.

The rule from the first section prevents all three: the expected value comes from the belief, worked out without the code. To check a new test, break the code on purpose. For example, make the function return 0 at once, and run the test again. If the test still passes, it does not test that code.

The test below gets want from lowest itself. The experiment runs it on the counterexample from the earlier section.

A test must be able to fail: its expected value comes from the belief, not from the code.

A test calls the code from outside, with inputs you chose. An assertion sits inside the code. It states that a condition is true at that point, every time the program reaches it. When the condition is false, the code itself is wrong, and the program should stop with a message.

Use assertions for programmer mistakes. A common one is a broken precondition: something that must be true when a function is called, such as “the list has at least one score”. The code that calls the function is responsible for it. Do not use assertions for problems that come from outside the program, such as a missing file, a score typed as text, or an empty list sent by another program. Those can happen when every line of the code is right. Handle them as errors and tell the user.

In Haskell, assert condition value, from Control.Exception, gives value when the condition is True, and throws an exception that stops the program when it is False. assert follows demand: the condition is checked only when something demands the result of assert. If nothing uses the result of lowest [], its condition is never checked. GHC removes every assert when it optimizes with -O, which most builds use, and when it is given the flag -fignore-asserts. For input from outside the program, return a Maybe or an Either and let the caller decide.

Assert what only a programmer mistake can break, and remember that -O removes the check. Return Maybe or Either for everything else.

Predict the output, then look for a counterexample before you reveal the answer. A team’s score sheet shows the top scorer of a game, by position in the team list. The belief is the comment at the top of the program. zip points [0 ..] pairs each score with its position, maximum picks the largest pair, and snd takes the position out of the pair.