Top-N and the Tail: your best-seller list depends on the metric, and hides most of the money

Before you start

New to this? A top-N list (a “leaderboard”) is just the first N rows after you sort a table by some column — “top 3 products”, “top 10 customers”. Two things about them catch nearly everyone. First, the column you sort by is a choice: sort products by units sold and by revenue (units × price) and you can get two lists with no products in common — the cheap thing everyone buys vs the dear thing a few people buy. Second, a short leaderboard shows the head of the distribution and quietly hides the tail — the “everyone else” rows — and the tail is often the bigger share of the total. This project teaches you to pick the ranking metric on purpose and to always report what the tail holds, so a top-N chart never fools you (or your boss).

1. The Brief

You’re handed one quarter of sales, one row per product, and asked the friendliest question in retail: what are our top 3 products? You can answer it in one line of pandas — and depending on whether you sort by units or by revenue, you hand back two completely different lists, each true, each pointing at a different decision. Worse, whichever top 3 you pick is the minority of the money: the other thirteen products — the tail nobody puts on a slide — are the bigger half of revenue. This is how a “top sellers” dashboard misleads: it answers an underspecified question and crops out most of the total. You’ll build both leaderboards, measure the head-vs-tail split, and learn to say which metric the question actually wanted.

2. Difficulty & time

Difficulty 2/10. One tool (pandas), one file, one concept — sort, slice a head, read a share — and Part A’s path is fully specified, so it sits at the low end of the scale. It is a notch below cohort-retention (3), silent-row-loss (3), and benford-fraud (3): each of those has more moving parts (a cohort matrix to pivot, a join to validate, an expected-vs-observed distribution to test), whereas this turns on a single habit — rank on purpose, then look at the tail. It stays off the very floor of the scale because it’s a judgment skill (which metric? what’s the tail worth?), not a syntax drill. Part A is a short read-and-run; Part B is where you do the work.

3. What you’ll be able to do after

Rank the same table two ways (by count and by value) and explain, in one sentence, why the top-N lists differ and which one a given decision needs.
Compute a top-N share of the total and the complementary “tail” share, and say which is bigger.
Spot when a “top sellers” / “top customers” report is hiding most of the total in its tail.
Answer “how many items make up 80% of the total?” from a cumulative share — not from a round number someone picked.

4. Prereqs & time box

Part A: ~30 minutes. Part B: ~1.5 hours. Both are hard caps — if Part A runs long, you’re overthinking it. You need to read a CSV with pandas, sort it, and do arithmetic on columns; nothing more.

For where to write and run Python (install, virtual environment, per-OS terminal, running a script step by step), see Start Here guide. No account and no GPU — requires_signup is empty.

5. Setup & data

The data is synthetic, generated locally by data/make_data.py from a fixed seed (byte-identical every run) — see Sources. Generate it once:

python data/make_data.py

That writes data/products_a.csv (Part A) and data/customers_b.csv (Part B). Nothing is downloaded; there is no external source. Synthetic is the right call here: the lesson is whether your leaderboard and your 80%-of-revenue count match the true distribution, and only generated data lets you know that truth in advance. The price/volume split (cheap staples vs big-ticket goods) and the heavy customer-spend tail are injected deliberately, and documented in the generator.

6. Part A — Guided Build

Prefer the browser? If this project shows a ▶ Run button, you can run Part A right on the page — no setup. The commands below are for running on your own computer.

▶ Try it now — run the real Part A right here in your browser. This one button makes the data and runs Part A for you — you do not need to install anything, and you do not need to type any of the python … commands anywhere on this page (those are only for running on your own computer instead). First run loads Python + this project’s libraries: usually ~10–20 s, a little longer for machine-learning projects.

Run the guided script (or paste it cell-by-cell — each # %% is one step):

python part_a/rank_products.py

It walks six steps and writes part_a/leaderboard_a.csv. The shape of the argument:

STEP 1. Load the 16 products. Checkpoint: prices span two orders of magnitude — a $650 espresso machine that sold in the hundreds, a $12 USB-C cable that sold in the thousands.
STEP 2. The leaderboard the buyer expects — top 3 by revenue. Checkpoint: Espresso Machine, Standing Desk, Ergonomic Chair (the dear, low-volume goods).
STEP 3. The head-vs-tail split. Checkpoint: total revenue about $1.51M; the top 3 are ~43.8% of it and the other 13 products — the tail — are ~56.2%. The tail is bigger.
STEP 4. Rank the same table by units sold — a different top 3 (USB-C Cable, Coffee Pods, A5 Notebook). Checkpoint: the two lists share no product; an assert proves it.
STEP 5. Both ranks side by side. Checkpoint: the espresso machine is #1 by revenue but #15 by units; the USB-C cable is the mirror image. That gap is the whole point.

Why this and not that

Why not just call the top 3 “our top products” and move on? Because “top” is undefined until you say by what. Units answers “what moves off the shelf / drives footfall and attach sales”; revenue answers “what pays the bills”. They’re different questions with different answers here — pick the one that matches the decision instead of letting a tool’s default (usually revenue) pick silently.
Why report the tail’s share at all? Because a top-N chart implies the rest is rounding error, and here the rest is the majority (56.2%). If you delisted the 13 tail products you’d lose more revenue than the top 3 make. The tail’s share is the number that keeps a leaderboard honest.

Functions you’ll meet (and where to read more)

Function	What it does here
`pandas.read_csv`	Load `products_a.csv` into a DataFrame
`DataFrame.sort_values`	Rank the products by revenue, then by units (STEP 2, 4)
`DataFrame.head`	Take the top N rows off a sorted table (STEP 2, 4)
`Series.rank`	Give each product its rank on both metrics (STEP 5)

If something looks off

If your two top-3 lists are the same, check you sorted ascending=False on the right column — by default sort_values sorts smallest-first.
If the top 3’s share comes out above 50%, you may have summed only the top 3’s revenue for both numerator and denominator — the denominator is all products’ revenue.
A number that surprises you is the point of this project, not a bug — read it against the checkpoints above before assuming you broke something.

7. Part B — Your Turn

Part B is the optional “your turn” half — a guided fill-in-the-blank, not a blank page. You get a working starter (part_b/starter.py) that already handles the easy version; your job is to finish the # TODO it marks out. More open than Part A’s step-by-step, but you are never staring at an empty file. If you finished Part A and want to prove the skill, this is where it happens. New to Python? It’s completely fine to stop after Part A and come back later.

Same skill, new setting. An online shop’s customers_b.csv gives each customer and how much they spent this quarter. Management keeps a “top 10 customers” report and treats those ten as “the customers who matter”. The starter reports 10 — and it’s wrong, because the top 10 are under half the revenue. The # TODO is the Part A move turned into a question: rank customers by spend, walk the cumulative share of revenue down the list, and find how many customers it takes to reach 80%. The round number “10” is not the answer; the cumulative curve decides.

Acceptance criteria (checkable)

Your part_b/starter.py writes part_b/out/report.json with:

n_customers_for_target — the number of top-spending customers whose cumulative revenue first reaches 80% of the total,
total_customers — the count of every customer in the file.

Then:

python part_b/starter.py
pytest part_b/test_solution.py -q

You’re done when the tests pass: total_customers is exact and n_customers_for_target matches the true cumulative-share crossing exactly (the reference hits it). The starter’s “top 10” guess is far too low — the top 10 are only ~44% of revenue — and fails.

Constraints — what must be true of your solution, not how to get there

The count must come from the running cumulative share of revenue, ranked high-to-low — not from a chosen round number and not from the total customer count.
Rank by spend descending before you accumulate; the first customer added is the biggest spender.

Hints

Hint 1 — what "cumulative share" means here

Sort customers by spend, biggest first. Walk down the list adding spend as you go; at each row you have “revenue from the top k customers”. Divide that running total by the grand total and you have the cumulative share at each rank.

Hint 2 — where the answer lives

The answer is the first rank where the cumulative share reaches 0.80. Everything above that rank is “the customers who make up 80% of revenue”; the round-number “top 10” stops well short of it.

Hint 3 — the shape of the answer

cum = ranked['spend'].cumsum() / ranked['spend'].sum(), then count the rows below the target and add one: n = int((cum < 0.80).sum() + 1). It should come out well above 10.

8. Self-check

Your n_customers_for_target should be much larger than 10 — the top 10 hold under half the revenue, so reaching 80% takes many more customers. If you got 10, you returned the starter’s guess, not the cumulative crossing.
Re-running python data/make_data.py never changes the data (it’s seeded), so your number shouldn’t move between runs.
If you can state, in one sentence, why the top 10 customers understate who the revenue depends on, you’ve got the transferable skill — the specific count is secondary.

9. Stretch

Add a Pareto/cumulative curve to Part B: for each rank plot the cumulative share of revenue, and mark where it crosses 50%, 80%, and 90%. Read off how many customers each milestone takes.
In Part A, compute the rank correlation between the units ranking and the revenue ranking (e.g. Spearman) to put a single number on “how much do the two leaderboards disagree”, and try to design a product mix where they’d agree.
(Genuinely hard.) The 80% cutoff is arbitrary. Show how n_customers_for_target moves as you sweep the target from 50% to 95%, and explain why the count climbs slowly at first and then steeply near the top — what shape of the spend distribution controls that curve, and what it would look like if every customer spent the same.

10. Ship it

Put this in a repo README so it counts as a portfolio piece: one sentence on the trap (“ranking by units and by revenue gave two disjoint top-3 lists, and the tail outside the top 3 was 56% of revenue”), the two leaderboards side by side, and the head-vs-tail split. Add a short note on Part B showing you transferred the same idea to a different question — from “what share do the top N hold” to “how many make up 80%” — and found that the “top 10 customers” report understated the base by 7.6×. Recognising the same skill under a different surface is what an interviewer is checking for.

11. Sources

See SOURCES.md. The data is synthetic, generated by data/make_data.py from a fixed seed; there is no external source, and nothing is downloaded or committed.

Next up

Finished this one? Continue the Data Analytics track:

Benford’s Law: catching fabricated numbers by their leading digit → · Browse all courses