Before you begin
Why this matters
When a program gives a wrong answer, the lines that cause it are usually a few among many. If you can shrink the program until only those lines are left, you can often see the cause yourself, and anyone you ask for help can run it in a minute.
What this helps you reason about
- Write a failure down precisely enough that someone else can make it happen.
- Shrink a failing program one deletion at a time without losing the failure.
- Notice when a change made the failure disappear for a reason that has nothing to do with the cause.
Case file: a shop shows each product’s average star rating rounded to a whole star, and the shop’s rule says halves round up. One product shows a star fewer than the rule gives. This lesson shrinks the program until the cause is the only thing left.
Already know this? The short version
- Write the expected result, the observed result, and the exact input before guessing; run the failure yourself.
- List explanations, each with an observation that would rule it out, and run the cheapest test first.
- Delete one piece of code or input at a time and keep a deletion only if the same wrong result remains; a rounder input such as 3.5 can hide the failure.
- Here the cause is round, which sends halves to the even neighbor (round 2.5 is 2); floor (x + 0.5) rounds positive halves up.
- Write it up with expected and observed results and the GHC version, make it print pass or FAIL, and keep it as a check after the fix.
Words in this lesson (5)
- reproduction
- A program and an input that make a failure happen every time they are run.
- hypothesis
- A possible explanation for a failure, stated so that some observation could show it is wrong.
- minimal reproduction
- A reproduction from which no single piece of code or input can be deleted without the failure disappearing.
- rounding half to even
- A rounding rule that sends a value exactly halfway between two whole numbers to the even one: 2.5 goes to 2 and 3.5 to 4. Haskell’s round follows it.
- regression test
- A check, kept with the code, that fails while a bug is present and passes once it is fixed, so the bug cannot come back unnoticed.
Case file · a star goes missing
- Expected
- An average of 2.5 shows 3 stars.
- Observed
- It shows 2 stars.
- Input
- The ratings 3, 2, 3, 2.
Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.
01 / The failing case
What exactly is failing?
A shop shows each product’s average rating rounded to a whole number of stars, and its rule says that halves round up. A product rated 3, 2, 3, and 2 has an average of 2.5 and should show 3 stars; it shows 2. Before guessing at a cause, write down three things: what you expected, what you saw, and the exact input. Together they are the failing case. Then run it yourself before changing anything. A failure you have seen happen is evidence. A failure someone only described to you may depend on something they did not mention, so it is not yet evidence.
Here the failing case is: expected 3, observed 2, input [3, 2, 3, 2]. In the program, fromIntegral turns each Int into a Double, so that / can divide them with a fraction.
average :: [Int] -> Double
average stars = fromIntegral (sum stars) / fromIntegral (length stars)
shown :: [Int] -> Int
shown stars = round (average stars)
-- shown [3, 2, 3, 2] is 2, expected 3Write the expected result, the observed result, and the exact input before you look for a cause.
Experiment
Run the failing case, then add a rating
average gives 2.5 for [3, 2, 3, 2], and round 2.5 is 2. With a fifth rating the average is 2.6, which is not a half, and round gives 3.
Why it matters A deletion after which the same wrong result remains shows the deleted piece is not needed for the failure. A change that makes the failure disappear shows something about what triggers it. Both are evidence, as long as you change one thing at a time.
Predict the stars shown for the ratings 3, 2, 3, 2. Then add one more rating, making the list [3, 2, 3, 2, 3], and predict again.
average :: [Int] -> Double
average stars = fromIntegral (sum stars) / fromIntegral (length stars)
shown :: [Int] -> Int
shown stars = round (average stars)
main :: IO ()
main = print (shown [3, 2, 3, 2])
What to look for
GHCi stops in shown and then several times in average, each time with stars = [3,2,3,2]. It shows one location per stop, not a stack of calls, and it does not show the average itself. The output is 2. After the edit, the average is 2.6 and the output is 3, the right answer: the failure appears when the average is exactly a half.
Expected output
202 / Explanations
Which explanations fit what you saw?
A hypothesis is a possible explanation, stated so that an observation could show it is wrong. List several before testing any: the first one you think of is often not the cause. For each, write what you would see if it were true, then run the cheapest test first. A test may change the program for a moment; undo the change before the next test. Here are three explanations for the missing star.
Test the competing explanations
Predict first. These are recorded outcomes of the example, not tests running in your browser.
01Print average [3, 2, 3, 2]
If the average is wrong, it is not 2.5.
Reveal the observation
Observed It prints 2.5.
What this rules in or out average is right; the problem is in the rounding.
02Print round 2.7
If round cuts off the fraction, as converting to a whole number does in many languages, it prints 2.
Reveal the observation
Observed It prints 3.
What this rules in or out round does not cut off the fraction.
03Print round 2.5 and round 3.5
If halves always round up, they print 3 and 4.
Reveal the observation
Observed They print 2 and 4.
What this rules in or out Halves go to the even neighbor: 2.5 down to 2, 3.5 up to 4.
An explanation is worth testing when some observation could rule it out.
03 / Reduce
What can you delete and still see the failure?
Reduce in small steps. Delete one piece: a function, a line, a field, an element of the input. Run again and compare the result with the failing case. If the same wrong result appears, keep the deletion. If the result changes, undo the deletion: that piece is part of the cause, and you have learned something about it.
Only delete while you reduce. Rewriting a line to make it simpler can change what the code does, and then a failure that disappears tells you nothing. Treat the input the same way: shorten it, and check every time that it still fails. You are done when no single deletion keeps the failure. That is a minimal reproduction.
Here average and the list of ratings go, and the input becomes the average itself, 2.5. The failure stays. :: Double tells the compiler which kind of number 2.5 is, and :: Int which kind round should give back; without them, it would have to choose for you.
main :: IO ()
main = print (round (2.5 :: Double) :: Int)
-- 2, expected 3Keep a deletion only if the same wrong result is still there.
Precisely: minimal is not smallest
Minimal here means that no single deletion keeps the failure. A different order of deletions can end at a different minimal program, and a cleverer rewrite might be shorter still. Minimal is enough: every line left is needed for the failure, so every line is a clue. Automatic tools that shrink inputs this way exist for many languages; the general method is called delta debugging.
Experiment
Pick another half and lose the failure
round sends a value exactly halfway between two whole numbers to the even one. For 3.5 the even neighbor is 4, the same as rounding up, so 3.5 cannot show the difference.
Why it matters An input where the wrong rule and the right rule agree cannot show the bug. When you pick a smaller or rounder input, check that it still fails.
Predict shown 2.5. Then replace 2.5 with 3.5 and predict again.
shown :: Double -> Int
shown average = round average
main :: IO ()
main = print (shown 2.5)
What to look for
GHCi stops in shown with average = 2.5, and the output is 2. After the edit, average = 3.5 and the output is 4, which looks right: rounding 3.5 up and rounding it to the even neighbor both give 4.
Expected output
204 / The rule
Which rule makes the smallest program fail?
Once the program is small, step through it and find the first line where a value differs from what you expected. Then look up the rule for that line in the documentation.
Here the first surprising value is round 2.5. The Haskell Report defines round to return the nearest whole number and, for a value exactly halfway between two, the even one: round 0.5 is 0, round 1.5 is 2, round 2.5 is 2. This is rounding half to even. Over many values it rounds up as often as down, which is why some kinds of accounting use it. The shop’s rule is different: halves round up. For positive values, floor (x + 0.5) follows that rule; floor gives the largest whole number that is not above its argument.
The bug is not in round: it does what the Report says. The bug was the assumption that round rounds halves up. Many reductions end like this, at a library function that behaves as documented.
shown :: [Int] -> Int
shown stars = floor (average stars + 0.5)
-- shown [3, 2, 3, 2] is 3The cause is the first line where a value differs from what you expected; the rule for that line explains why.
Coming from JavaScript: Math.round(2.5) is 3 in JavaScript, and Math.round(2.5) is 3 in Java too; Haskell’s round 2.5 is 2.
Coming from Python: Python 3’s round(2.5) is also 2: it rounds halves to even as well.
Precisely: negative halves and stored fractions
floor (x + 0.5) rounds -2.5 to -2. A rule that rounds halves away from zero would give -3; write it with signum and abs if you need it. Also, many decimal fractions are not stored exactly in a Double: 2.675 is stored as slightly less, so a value that looks like a half after a calculation may not be one. 2.5 itself is stored exactly.
Experiment
Round four halves
round x picks the even whole number when x is exactly a half. floor (x + 0.5) moves every half up to the next whole number and then drops what is left of the fraction.
Why it matters To find the rule a function follows, run it on a short list of boundary values, such as every half from 0.5 to 3.5, and compare the results with the rule you assumed.
Predict the list for 0.5, 1.5, 2.5, and 3.5. Then change nearest x = round x to nearest x = floor (x + 0.5) and predict again.
nearest :: Double -> Int
nearest x = round x
main :: IO ()
main = print (map nearest [0.5, 1.5, 2.5, 3.5])
What to look for
GHCi stops in nearest once for each element, and x shows the half being rounded. The output grows as the list is printed: [ appears before the first number is computed, then [0, and so on, because print writes each part as soon as it is demanded. The output is [0,2,2,4]. After the edit, GHCi stops twice for each value, and the output is [1,2,3,4].
Expected output
[0,2,2,4]05 / Write it up
Can someone else run your reproduction?
A reproduction is finished when someone else can run it and see the failure without asking you anything. Write four things: what you expected, what happened, the program, and where you ran it (the language version and the system). Make the program print its own verdict, pass or FAIL, so a reader does not need to know the right answer to read the output.
After the fix, keep the same program as a regression test. A regression is a bug that comes back after it was fixed, and a regression test is a check that fails while the bug is present and passes once it is fixed. If someone later brings the bug back, the check prints FAIL again.
-- Expected 3, observed 2. GHC 9.0.2.
main :: IO ()
main = do
let got = round (2.5 :: Double) :: Int
putStrLn (if got == 3 then "pass" else "FAIL: got " ++ show got)A reproduction prints its own verdict, so anyone can run it and tell whether the bug is still there.
Why is this a better reproduction to send than the program you started with?
Reveal the reasoning
It has nothing left to delete: no ratings, no average, and one value, 2.5. A reader sees at once that round is the question and does not have to work out an average. It also prints FAIL without the reader knowing the right answer. After the fix the shop no longer uses round for this, so the regression check calls shown [3, 2, 3, 2] and expects 3.
06 / Try it
Can you shrink a fresh failure?
Predict the output, then plan the reduction before you reveal the answer. A survey report should show each answer as a whole percentage, with halves rounded up.
percent :: Int -> Int -> Int
percent part total =
round (100 * fromIntegral part / fromIntegral total :: Double)
main :: IO ()
main = do
putStrLn "Survey results"
print (percent 1 8)
print (percent 3 8)
What does this print, which lines can you delete and still see the failure, and what is the smallest input that still fails?
Reveal the reasoning
It prints Survey results, then 12 and 38. One eighth is 12.5 percent, which the rule rounds up to 13; round gives 12, the even neighbor. Three eighths is 37.5, and round gives 38, which matches the rule, so that line cannot show the failure. You can delete the title, the second line, and percent itself. The smallest reproduction is round 0.5, which gives 0 where the rule gives 1. The fix is floor (x + 0.5) in place of round.
Practice tickets
- REPRO-101 · Fix the longest streak → Free
- REPRO-102 · Pin the parking charge with a test → Sign in · Pro
- REPRO-103 · Shrink a crashing formula → Sign in · Pro
Keep exploring
Assertions, tests, and counterexamples →