Your First Test: catch a real bug by testing the boundary, not the happy path

Before you start

New to this? First, two words. A function is a small, reusable rule that takes something in and gives one answer back — think of a spreadsheet formula with a name: give discount an order quantity and it hands you the price. A test is just code that runs that function on a chosen input and checks the answer is what you expected — e.g. “discount(10) should return 90; is it?” If it isn’t, the test fails and you’ve caught a bug (the function returning the wrong answer). The trap this project is about: a function can be correct on every “normal” input and wrong on exactly one — a boundary, the value where its behaviour switches (a discount kicks in, a price bracket changes). Test three comfortable middle-of-the-road inputs and everything passes, so the bug ships. Test the exact edge and it falls over. This project is not about pytest’s syntax (that’s a few lines you’ll pick up in minutes) — it’s about the judgment of which inputs are worth testing. The answer, almost always, is: the boundaries.

1. The Brief

You’re handed a small pricing function that a teammate wrote and “tested” by trying a few orders. It looked fine, so it shipped. It is still wrong — it overcharges every customer whose order lands exactly on a discount tier’s edge, because the discount starts one unit too late. You’ll run the buggy function and a correct spec side by side over a week of real orders, watch the two disagree on precisely the boundary rows (and nowhere else), and see why the teammate’s happy-path check could never have caught it. Then, in Part B, you’ll do the catching yourself on a different function with a different edge bug. This is the single most useful testing habit there is: bugs cluster at boundaries, so that’s where you aim. A team that only tests the happy path ships boundary bugs to production — invisibly, until a customer notices.

2. Difficulty & time

Difficulty 2/10. One concept (test the boundaries), one small pure function at a time, and Part A’s path is fully specified — the low end of the scale. It sits above missing-not-random (1): that one is a single unavoidable observation about one dataset, whereas this asks you to generate the revealing inputs yourself in Part B. It sits below data-contracts (3), which layers several rule types and a cross-field check, and well below property-testing (6) — this is the gentle, example-by-hand introduction to the same instinct that project automates: where property-testing has the Hypothesis library search for a violating input, here you pick the boundary inputs by eye. Getting the judgment by hand first is exactly what makes the automated version make sense later. Anchored to the rubric’s 1–2 band and those named ledger projects (G10).

Time is separate from difficulty: Part A runs in well under a second (measured Part A wall-clock 0.38s). Part A is about 30 minutes of reading and running; Part B is ~1.5 hours of your own work.

3. What you’ll be able to do after

Given a function with tiers, ranges, or thresholds, name its boundaries and write a test case on each one — the inputs where a bug is most likely to hide.
Explain, in one sentence, why a “one normal input per branch” test can pass while the function is still broken.
Write a test whose expected value comes from the spec, not from what the code happens to return (so the test can actually fail).
Tell a test that would catch a boundary bug from one that only looks thorough.

4. Prereqs & time box

Part A: ~30 minutes. Part B: ~1.5 hours. Both are hard caps. You need to write a Python function and compare two numbers; that’s all. No prior testing experience is assumed — the one pytest idea you need (an assert that checks a value) is introduced where you use it.

For where to write and run Python (install, virtual environment, per-OS terminal, running a script step by step), see Start Here guide. No account and no GPU — requires_signup is empty.

5. Setup & data

The data is synthetic, generated locally by data/make_data.py from a fixed seed (byte-identical every run) — see Sources. Generate it once:

python data/make_data.py

That writes data/cases_a.csv (a week of order quantities, for Part A) and data/cases_b.csv (a day of parcel weights, for Part B). Nothing is downloaded; there is no external source. Synthetic is the right call here: the whole lesson is does your test catch the planted bug, and only generated data lets you plant a bug at a known boundary and check that you found it. One deliberate detail — cases_a.csv contains the boundary quantities so Part A can show you the bug, while cases_b.csv avoids every boundary so Part B’s bug stays hidden until you go looking for it.

6. Part A — Guided Build

Prefer the browser? If this project shows a ▶ Run button, you can run Part A right on the page — no setup. The commands below are for running on your own computer.

▶ Try it now — run the real Part A right here in your browser. This one button makes the data and runs Part A for you — you do not need to install anything, and you do not need to type any of the python … commands anywhere on this page (those are only for running on your own computer instead). First run loads Python + this project’s libraries: usually ~10–20 s, a little longer for machine-learning projects.

Run the guided script (or paste it cell-by-cell — each # %% is one step):

python part_a/find_the_bug.py

It walks six steps and writes part_a/disagreements_a.csv. The shape of the argument:

STEP 1–2. Write the discount as a correct function from the spec (1–9 tubes: full price; 10–49: 10% off; 50+: 20% off), then read the shipped function beside it. The only difference is > where the spec means >= — the discount starts one tube too late.
STEP 3. Run both over the week’s orders. Checkpoint: 44 orders load, and on most of them the two functions agree — the bug is invisible on ordinary quantities.
STEP 4. Find where they disagree. Checkpoint: exactly 2 rows — the order at quantity 10 (shipped $100.00, should be $90.00) and the order at quantity 50 (shipped $450.00, should be $400.00). Both are tier boundaries.
STEP 5. Confirm the boundary’s neighbours (9, 11, 49, 51) all agree — so a tester who picked “a small, a medium, a big order” (say 5, 30, 80) would have seen nothing wrong.
STEP 6. The rule: for tiers, ranges, thresholds, or cut-offs, test the exact edge and one step either side. Checkpoint: the bug hides on 42 rows and shows on 2.

Why this and not that

Why not just test one order per tier (5, 30, 80)? Because that covers the values but not the behaviour. The function’s behaviour only changes at 10 and 50; an input that isn’t near an edge exercises the same branch as its neighbours and tells you nothing new. Coverage of values is not coverage of behaviour.
Why compute the expected price from the spec, not from the function? If you check the code against itself (“does it return what it returns?”) the test can never fail. A real test states what the answer should be, independently — here, from the written discount rule.
Why does the bug hide at all? Because real order quantities rarely land on an exact tier edge, so day to day the buggy and correct functions return the same number. That’s precisely why boundary bugs survive casual testing and reach customers.

Functions you’ll meet (and where to read more)

Function	What it does here
`pandas.read_csv`	Load `cases_a.csv` into a DataFrame
`Series.apply`	Run the pricing function on every order quantity
`DataFrame.to_csv`	Save the side-by-side comparison to `disagreements_a.csv`
`assert`	The one statement a test is built on: check a value, fail loudly if it’s wrong
`pytest`	The test runner you’ll use in Part B to run your `assert`s

If something looks off

If zero rows disagree, check that cases_a.csv was generated (python data/make_data.py) — it deliberately includes one order at quantity 10 and one at 50.
If many rows disagree, your correct function probably doesn’t match the spec — re-read the tier bounds (10–49 inclusive gets 10% off; 50 and up gets 20%).
A surprising number is the point of this project, not a bug — read it against the checkpoints above before assuming you broke something.

7. Part B — Your Turn

Part B is the optional “your turn” half — a guided fill-in-the-blank, not a blank page. You get a working starter (part_b/starter.py) that already runs; your job is to replace its weak test cases with ones that actually catch the bug. More open than Part A’s step-by-step, but you are never staring at an empty file. If you finished Part A and want to prove the skill, this is where it happens. New to Python? It’s completely fine to stop after Part A.

Same skill, new function. A warehouse prices parcels by weight bracket:

Weight (kg)	Fee
over 0, up to and including 1	`$3`
over 1, up to and including 5	`$7`
over 5, up to and including 20	`$15`
over 20	`$40`

The shipped buggy_shipping_fee has a boundary bug (a < where the spec means <=), so a parcel weighing exactly a bracket edge is charged the next bracket up. The starter runs that function over a day of real parcels — and everything looks fine, because not one real parcel weighs an exact edge. Your job: choose the test cases that expose the bug. The twist versus Part A: the data won’t hand you the failing rows this time. You have to read the spec, work out where the edges are, and write those cases yourself.

Acceptance criteria (checkable)

Your part_b/starter.py writes part_b/out/report.json containing:

test_cases — a list of [weight_kg, expected_fee] pairs you chose, where each expected_fee is what the spec table above says (compute it yourself, don’t call the buggy function),
achieved_metric — the fraction of the injected boundary bugs your cases catch.

Then:

python part_b/starter.py
pytest part_b/test_solution.py -q

You’re done when both tests pass. The grader re-computes the metric from your raw test_cases (it doesn’t trust the number you wrote): every case must be valid (its expected fee matches the spec) and together they must catch every boundary bug — a metric of 1.0 against a published bar of 0.99. The shipped happy-path starter catches 0 and fails; the reference catches all three edges (1.0).

Constraints — what must be true of your solution, not how to get there

Every expected_fee you write must come from the spec table, not from what any function returns. A test that agrees with the buggy code is worse than no test.
You must catch the bug at every bracket boundary, not just one. Thoroughness is the point.
Don’t delete the starter’s read of cases_b.csv — seeing the bug stay hidden on real data is half the lesson.

Hints

Hint 1 — where does a bracket function change behaviour?

At its edges — the exact weights where one fee becomes the next. Read them straight off the spec table: 1, 5, and 20 kg. A weight in the middle of a bracket (like 3 or 12) can’t reveal a boundary bug, because the boundary is somewhere else.

Hint 2 — what's the expected fee at an edge?

The table’s upper bound is inclusive (“up to and including”). So a 1.0 kg parcel is still in the first bracket → $3; a 5.0 kg parcel is in the second → $7; a 20.0 kg parcel is in the third → $15. Those are the values the buggy < gets wrong.

Hint 3 — the shape of the answer

TEST_CASES = [(1.0, 3), (5.0, 7), (20.0, 15)]. Each pair is (an exact edge weight, the fee the spec assigns it). Run the buggy function on those weights and you’ll see it return 7, 15, 40 instead — three caught bugs, metric 1.0.

8. Self-check

Your three edge cases should return the spec fee under the correct reading and a higher fee under the buggy one — that gap is the bug you caught.
If your metric is 0.67, you missed one edge; if it’s 0.0, none of your weights landed on a boundary — you tested the middle of the brackets, like the starter.
Re-running python data/make_data.py never changes the data (it’s seeded), so your result shouldn’t move between runs.
If you can state, in one sentence, why you test the edges and not the middle, you’ve got the transferable skill — the specific weights are secondary.

9. Stretch

Turn your Part B cases into a real pytest file: write one def test_...() per edge that asserts the spec fee, e.g. assert correct_shipping_fee(1.0) == 3. Point those asserts at the fixed function and confirm they pass; point the same asserts at buggy_shipping_fee and watch them fail. A good test suite goes green on correct code and red on broken code — that’s the whole contract of a test.
Add the just-inside neighbours (0.9, 4.9, 19.9 kg) and the just-outside ones (1.1, 5.1, 20.1) to your cases and confirm they all agree — proving the bug is a knife-edge, exactly one value wide, and building the habit of bracketing an edge from both sides.
(Genuinely hard.) Without being told where the bug is, write a small routine that finds every boundary of an unknown bracket function automatically: sweep the weight axis finely and flag each weight where the fee changes, then test a case on each. That is the seed of what property-testing (ledger, difficulty 6) does with the Hypothesis library — you’ve just built the manual version.

10. Ship it

Put this in a repo README so it counts as a portfolio piece: one sentence on the trap (“the shipped function overcharged every order sitting exactly on a discount tier edge”), the two disagreeing rows side by side, and the one-line rule (“test the boundary, not the happy path”). Add a short note on Part B showing you transferred the same instinct to a different function with a different edge bug — and that you derived the failing inputs from the spec when the data didn’t contain them. Recognising the same skill under a different surface is what an interviewer is actually checking for.

11. Sources

See SOURCES.md. The data is synthetic, generated by data/make_data.py from a fixed seed; there is no external source, and nothing is downloaded or committed.

Next up

Finished this one? Continue the Open-Source Libraries track:

Analytical SQL with DuckDB: from clickstream to sessions → · Browse all courses