Week-Over-Week Change When the Numbers Are Small

A product goes from two users to six in a week. That is a rise of two hundred per cent, and it looks magnificent on a chart. The following week it goes from six to five, a fall of seventeen per cent, which looks like a problem. In truth, both changes are the movement of a person or two. Nothing about the product necessarily changed.

Small numbers behave differently from large ones, and the habits that serve a big company can mislead a small one. This guide shows how to read week-over-week change when your counts are low, how to report it without exaggeration and how to use it as an honest guide to what to do next.

What week-over-week change is

Week-over-week change compares a measure in one week with the same measure in the previous week. It is usually expressed as a percentage: the difference divided by the earlier figure. Relative change is the standard name for a difference expressed in proportion to a starting value.

For example, from forty to fifty is plus ten, and ten divided by forty is twenty-five per cent. The calculation is simple, and it is useful because it lets you compare things of different sizes.

It is also the source of the problem. The percentage depends heavily on the starting value. The smaller the base, the larger the percentage that a small absolute change produces.

Why small numbers are slippery

Two ideas help to explain this.

Sampling error. Any count of people in a short period is, in effect, a sample of the people who could have shown up. The difference between what you observe and what would be observed over a long time is sampling error, and it is larger when the sample is small. In a week where eight people arrive, the real underlying rate might be six or twelve.

The law of small numbers. This name, used half humorously by psychologists, describes the mistaken belief that small samples will resemble the larger population they come from. People expect a handful of observations to look like the long run, and are surprised, or excited, when they do not. A small sample can look striking by chance alone.

Together, these mean that a large percentage from a small base is weak evidence of anything.

A quick demonstration

Imagine a product that, on average, gains five users a week, with ordinary random variation. Over six weeks it might record: 4, 7, 3, 6, 8, 2.

The week-over-week changes are plus 75 per cent, minus 57 per cent, plus 100 per cent, plus 33 per cent and minus 75 per cent. These swings look wild. Yet nothing changed in the underlying rate. The chart of percentages looks like a drama, and the chart of counts looks like a small business going along steadily.

The lesson: if your weekly figure is under, say, a few dozen, the percentage change will often be noise.

Show the counts

The simplest and best remedy is to show the numbers behind the percentage.

• Write "from 4 to 7 sign-ups, plus 75 per cent", not just "plus 75 per cent".
• Put the counts in your charts and tables.
• Say the period: "this week versus last week".
• If the base is small, say so: "a small base, so treat with caution".

Readers can then see the scale and decide how much weight to give it. Hiding the count to make the percentage look better is the quickest way to lose trust.

Use more than two points

A comparison of two weeks tells you little. A short series tells you more. With only a handful of weeks, you still can do useful things.

Plot the counts, in order, on a simple line or bar chart.

Add a moving average, a calculation that averages the last few periods, say three weeks, and recalculates each week. A moving average smooths short-term variation, so that the direction is easier to see. In the earlier example, a three-week moving average of the counts would hover between four and six, correctly showing a flat line.

Look at the cumulative total, which smooths even more and shows overall progress.

Compare longer spans: four weeks against the previous four. Larger blocks reduce noise.

Use the median in place of the mean if one week is unusually high or low. The median is the middle value when the figures are ordered, and it is less affected by extremes.

Resist drawing a conclusion from fewer than four or five weeks.

When a change is probably real

No simple rule exists, but a few signs add confidence.

• The change is large in absolute terms, not just in proportion.
• It persists for several periods, not one.
• It lines up with something you did, such as a release or a campaign, and you have a reason to expect the effect.
• Independent measures agree, such as sign-ups, visits and returning users all rising.
• You can see the people, because with small numbers you can read each one's story.

At small scale, you have an advantage that large companies lack: you can look at every user. If six people signed up, you can check who they are, where they came from and what they did. A short look at the individuals often tells you more than a chart.

Reporting honestly

When you share numbers publicly, a few habits protect you.

• Give the count and the period with every percentage.
• Avoid headline percentages from tiny bases. "Up 300 per cent" from two to eight is technically true and likely to mislead.
• Use words that match the evidence. "A promising few weeks" is better than "explosive growth".
• Show the series, not just the best week.
• Say what you did not measure.
• Correct yourself if later data shows that an early reading was a blip.

Honest reporting is also good for you. Accurate numbers guide better decisions.

Reading other people's numbers

The same care applies when you read a board or a post.

• Ask for the base. A big percentage from an unknown count is a headline, not information.
• Look at the period. A week is short.
• Check consistency. Does the product appear at the top once, or repeatedly?
• Compare like with like, in category and size.
• Be wary of rank changes among products with small numbers, where one person can move several places.

A board that shows the counts alongside the changes is easier to trust. One that shows only percentages deserves more questions.

Alternatives to percentages

When the base is small, other ways of describing progress can be clearer.

• Counts per week, shown in a row.
• People, by name, in the sense of knowing and describing each early user.
• Milestones: "the first ten returning users", "the first paying customer".
• Retention: of the last ten sign-ups, how many came back?
• Qualitative evidence: quotations, stories, requests.

A founder who can say "seven of the last ten who signed up returned in the following week, and here is why" has given a more useful account than one who says "growth is up forty per cent".

What to do with the signal

Use small-number data to guide actions, but hold conclusions loosely.

• If counts rise steadily over several weeks, keep doing what you are doing.
• If counts fall, check for a cause: a broken link, a changed page, a seasonal dip.
• If counts are erratic, do not change anything on the basis of one week.
• If counts are flat, that is information too. Consider an experiment with a clear hypothesis.

Because the data is noisy, combine it with conversations. Numbers say what, and people say why.

A worked example

A maker of a small habit-tracking app records weekly new sign-ups: 6, 9, 5, 12, 8, 11. His analytics dashboard flags the third week as minus 44 per cent and the fourth as plus 140 per cent. He feels a surge of worry and then of hope.

He plots the counts and adds a three-week moving average. The average goes from about 7 to about 9, and the line rises gently. The percentages that excited and alarmed him were mostly noise around a modest upward drift. He looks at the individuals in week four and finds that five of the twelve came from a single forum post. Without that, the week would have been seven.

He writes in his notes: "Roughly 8 to 9 a week, a gentle rise. One forum post added about five in week four." When he shares an update with users, he says that sign-ups have risen from around seven a week to around nine over the last six weeks, with one week boosted by a forum post. The statement is modest and true. It also gives readers something to check, and when the numbers continue to rise, his credibility rises with them.

Questions makers ask

At what size do percentages become reliable? There is no sharp line, but with weekly counts in the hundreds, percentages begin to behave better. Under a few dozen, treat them as rough.

Should I stop using percentages? No. Use them, but with the count and the period beside them.

What if a platform shows only percentages? Note the base yourself when you discuss it.

How long until the numbers settle? As your counts grow, so does reliability. Keep tracking and the picture sharpens.

Summary

Week-over-week percentages are volatile when the base is small, because of sampling error and the tendency to read patterns into little data. Show the counts and the period alongside every percentage, plot several weeks, add a moving average and compare longer spans. Look for changes that are large in absolute terms, persistent, explainable and echoed by other measures. Report in words that match the evidence and read other people's figures with the same care. With small numbers, the best data may be the stories of the people themselves.