Before you begin
Why this matters
When a program gives a wrong answer, the lines that cause it are usually a few among many. If you can shrink the program until only those lines are left, you can often see the cause yourself, and anyone you ask for help can run it in a minute.
What this helps you reason about
- Write a failure down precisely enough that someone else can make it happen.
- Shrink a failing program one deletion at a time without losing the failure.
- Notice when a change made the failure disappear for a reason that has nothing to do with the cause.
Case file: a race results page should show the fastest time first, and it shows the wrong one. Nothing crashes. This lesson shrinks the program until the cause is the only thing left.
Already know this? The short version
- Write the expected result, the observed result, and the exact input before guessing; run the failure yourself.
- List explanations, each with an observation that would rule it out, and run the cheapest test first.
- Delete one piece of code or input at a time and keep a deletion only if the same wrong result remains; do not rewrite while reducing, and do not swap in “simpler” input without checking it still fails.
- Here the cause is sort() with no arguments: it compares elements as text, so [10, 9] stays [10, 9]; pass (a, b) => a - b to compare numbers.
- Write it up with expected and observed results and the Node.js version, make it print pass or FAIL, and keep it as a check after the fix.
Words in this lesson (5)
- reproduction
- A program and an input that make a failure happen every time they are run.
- hypothesis
- A possible explanation for a failure, stated so that some observation could show it is wrong.
- minimal reproduction
- A reproduction from which no single piece of code or input can be deleted without the failure disappearing.
- compare function
- A function passed to sort that receives two elements and returns a negative number when the first should come first, a positive number when the second should, and 0 when their order does not matter.
- regression test
- A check, kept with the code, that fails while a bug is present and passes once it is fixed, so the bug cannot come back unnoticed.
Case file · the fastest time is wrong
- Expected
- The fastest of 12, 9, and 10 s is 9.
- Observed
- The page shows 10.
- Input
- The times [12, 9, 10], in that order.
Examples are focused snippets; imports, enclosing functions, or classes may be omitted. C examples use C11; Haskell examples use Haskell 2010. Expand the notes for language-specific depth.
01 / The failing case
What exactly is failing?
The results page should show the fastest time first. With the times 12, 9, and 10 seconds, it shows 10. Before guessing at a cause, write down three things: what you expected, what you saw, and the exact input. Together they are the failing case. Then run it yourself before changing anything. A failure you have seen happen is evidence. A failure someone only described to you may depend on something they did not mention, so it is not yet evidence.
Here the failing case is: expected 9, observed 10, input [12, 9, 10]. In the program, times.slice() makes a copy of the array, and sorted.sort() puts the copy’s elements in order, changing the copy itself.
function fastest(times: number[]): number {
const sorted = times.slice(); // a copy; times keeps its order
sorted.sort();
return sorted[0];
}
console.log(fastest([12, 9, 10])); // 10, expected 9Write the expected result, the observed result, and the exact input before you look for a cause.
Experiment
Run the failing case, then delete a part
fastest copies the times, sorts the copy, and returns its first element. The copy comes out of sort as [10, 12, 9], so the first element is 10. label only formats the number it receives.
Why it matters A deletion after which the same wrong result remains shows the deleted piece is not needed for the failure. A change that makes the failure disappear shows something about what triggers it. Both are evidence, as long as you change one thing at a time.
Predict what the program prints. Then make the first deletion: replace console.log(label("fastest", fastest(times))) with console.log(fastest(times)) and run again. Is the failure still there?
// The fastest race time should come first.
function fastest(times: number[]): number {
const sorted = times.slice();
sorted.sort();
return sorted[0];
}
function label(name: string, seconds: number): string {
return name + ": " + seconds + " s";
}
const times = [12, 9, 10];
console.log(label("fastest", fastest(times)));
What to look for
Step into fastest and watch sorted. After sorted.sort() runs, sorted changes from [12, 9, 10] to [10, 12, 9], and the output is fastest: 10 s. After the edit, the output is 10: the same wrong value, so label was not needed for the failure. The top-level frame also lists fastest and label, shown as function fastest(times) and function label(name, seconds): the panel lists a declared function as a variable, by its name and parameters. Step sometimes stops on the same line two or three times; that is normal for this debugger.
Expected output
fastest: 10 s02 / Explanations
Which explanations fit what you saw?
A hypothesis is a possible explanation, stated so that an observation could show it is wrong. List several before testing any: the first one you think of is often not the cause. For each, write what you would see if it were true, then run the cheapest test first. A test may change the program for a moment; undo the change before the next test. Here are three explanations for the wrong time.
Test the competing explanations
Predict first. These are recorded outcomes of the example, not tests running in your browser.
01Print fastest(times) without label
If label is at fault, for example by rounding or reformatting the number, the bare result is 9.
Reveal the observation
Observed The bare result is 10.
What this rules in or out label is not the cause; the wrong number comes from fastest.
02Print times before calling fastest
If the times arrived as text, such as "12" from a form or a JSON file, or 9 is missing, times does not show the three numbers.
Reveal the observation
Observed times is [12, 9, 10], three numbers.
What this rules in or out The input is right, so the problem is inside fastest.
03Print sorted after sorted.sort()
If sort orders the numbers wrongly, 9 is not first.
Reveal the observation
Observed sorted is [10, 12, 9].
What this rules in or out sort() is where the order goes wrong. The next step is to find out why.
An explanation is worth testing when some observation could rule it out.
03 / Reduce
What can you delete and still see the failure?
Reduce in small steps. Delete one piece: a function, a line, a field, an element of the input. Run again and compare the result with the failing case. If the same wrong result appears, keep the deletion. If the result changes, undo the deletion: that piece is part of the cause, and you have learned something about it.
Only delete while you reduce. Rewriting a line to make it simpler can change what the code does, and then a failure that disappears tells you nothing. Treat the input the same way: shorten it, and check every time that it still fails. You are done when no single deletion keeps the failure. That is a minimal reproduction.
Here the label function goes first, then fastest itself: sorting the array directly keeps the failure. The input shrinks to two numbers, [10, 9]. It cannot shrink to one: an array of one number is always in order.
const times = [10, 9];
times.sort();
console.log(times); // [ 10, 9 ], expected [ 9, 10 ]Keep a deletion only if the same wrong result is still there.
Precisely: minimal is not smallest
Minimal here means that no single deletion keeps the failure. A different order of deletions can end at a different minimal program, and a cleverer rewrite might be shorter still. Minimal is enough: every line left is needed for the failure, so every line is a clue. Automatic tools that shrink inputs this way exist for many languages; the general method is called delta debugging.
Experiment
Change the input and lose the failure
sort() orders "10", "12", and "9" as text, where "1" comes before "9". When every number has one digit, text order and number order agree, so the failure needs at least one number with more digits.
Why it matters [7, 5, 8] looks like a smaller version of [12, 9, 10], but it has no number with two digits, which is what the failure needs. Two numbers such as [10, 9] keep that property.
Predict the order sort() gives. Then replace [12, 9, 10] with [7, 5, 8], numbers that are easier to read, and predict again.
const times = [12, 9, 10];
times.sort();
console.log(times);
What to look for
Watch times in the variables panel. After times.sort() it shows [10, 12, 9], and the output is [ 10, 12, 9 ]. After the edit, it shows [5, 7, 8], which is correct: the failure is gone.
Expected output
[ 10, 12, 9 ]04 / The rule
Which rule makes the smallest program fail?
Once the program is small, step through it and find the first line where a value differs from what you expected. Then look up the rule for that line in the documentation.
Here the line is times.sort(). Called with no arguments, sort turns each element into text and compares the texts one character at a time, the way a dictionary orders words. "10" comes before "9" because "1" comes before "9". To sort numbers as numbers, pass a compare function: a function that receives two elements and returns a negative number when the first should come first. (a, b) => a - b is such a function, written with an arrow: a and b are its parameters, and a - b is the value it returns. For a = 9 and b = 10, a - b is -1, a negative number, so 9 goes before 10.
The bug is not in sort: it does what its documentation says. The bug was the assumption that sort compares numbers. Many reductions end like this, at a library function that behaves as documented.
const times = [10, 9];
times.sort((a, b) => a - b);
console.log(times); // [ 9, 10 ]The cause is the first line where a value differs from what you expected; the rule for that line explains why.
Coming from Python: Python’s sorted([10, 9]) compares the numbers as numbers and gives [9, 10]; JavaScript’s sort() compares their text unless you pass a compare function.
Coming from Java: Java’s Arrays.sort on an int[] sorts by value because the array’s type says int; a JavaScript array has no element type while it runs, so sort() falls back to text.
Precisely: what sort compares
ECMAScript specifies that sort without a compare function converts elements to strings and compares them by UTF-16 code units, with undefined elements placed last. Since ES2019 the sort is stable: elements that compare as equal keep their order. Typed arrays such as Float64Array sort numerically without a compare function. TypeScript’s type number[] does not change any of this, because types are removed before the program runs.
Experiment
Watch sort compare text, then numbers
Without a compare function, sort turns 9 into "9" and 10 into "10" and compares the texts one character at a time. With a compare function, sort calls it for pairs of elements and uses the sign of the result: negative puts a first.
Why it matters When a call gives a surprising result, test the rule you think it follows on the smallest values you can, such as "9" < "10".
Predict whether nine < ten is true. Then replace times.sort() with times.sort((a, b) => a - b) and predict the new order.
const nine = String(9);
const ten = String(10);
console.log(nine < ten);
const times = [12, 9, 10];
times.sort();
console.log(times);
What to look for
The first line prints false: as text, "9" comes after "10". times changes from [12, 9, 10] to [10, 12, 9]. After the edit, the stepper stops on the sort line several times, in a new frame on top with its own a and b, such as a=9, b=12. That frame is the arrow function; it has no name, so the panel names it arrow function (line 5), after the line where it is written. While the comparisons run, times still shows [12, 9, 10]: the panel shows the new order only after sort has finished. times ends as [9, 10, 12].
Expected output
false
[ 10, 12, 9 ]05 / Write it up
Can someone else run your reproduction?
A reproduction is finished when someone else can run it and see the failure without asking you anything. Write four things: what you expected, what happened, the program, and where you ran it (the language version and the system). Make the program print its own verdict, pass or FAIL, so a reader does not need to know the right answer to read the output.
After the fix, keep the same program as a regression test. A regression is a bug that comes back after it was fixed, and a regression test is a check that fails while the bug is present and passes once it is fixed. If someone later brings the bug back, the check prints FAIL again.
In the program below, condition ? a : b gives a when the condition is true and b otherwise.
// Expected: 9 first. Observed: 10 first. Node.js 22.
const times = [10, 9];
times.sort();
console.log(times[0] === 9 ? "pass" : "FAIL: first is " + times[0]);A reproduction prints its own verdict, so anyone can run it and tell whether the bug is still there.
Why is this a better reproduction to send than the program you started with?
Reveal the reasoning
It has nothing left to delete: no label, no fastest, and two numbers instead of three. A reader sees at once which two values sort wrongly and does not have to wonder whether 12 or the label matters. It also prints FAIL without the reader knowing the right answer, and after the fix it prints pass.
06 / Try it
Can you shrink a fresh failure?
Predict the output, then plan the reduction before you reveal the answer. Bookmarks in a reading app should be listed from the first page to the last.
const title = "Field guide";
const pages = [100, 25, 5];
pages.sort();
console.log(title);
for (const page of pages) {
console.log("page " + page);
}
What does this print, which lines can you delete and still see the failure, and what is the smallest input that still fails?
Reveal the reasoning
It prints Field guide, then page 100, page 25, page 5: the order is backwards. You can delete the title, its print, and the loop, and print pages directly; the same wrong order stays. The input can shrink to two numbers whose text order differs from their number order, such as [10, 5]: sort() keeps it as [10, 5]. It cannot shrink to one number, and [5, 60] would not fail, because "5" comes before "60" as text too. The fix is pages.sort((a, b) => a - b).
Practice tickets
- REPRO-101 · Fix the longest streak → Free
- REPRO-102 · Pin the parking charge with a test → Sign in · Pro
- REPRO-103 · Shrink a crashing formula → Sign in · Pro
Keep exploring
Assertions, tests, and counterexamples →