The Long Run
An early result is not an estimate. Six customers average $2,900 against a $4,000 target. The marketer waits for more data, and Dr. Bayes explains what actually improved.
Behind the joke.
An average of six customers is very noisy: one large or small order moves it a lot. As the sample grew to 30 and then 100 customers, the average moved from $2,900 to $3,600 to $4,050 and became more reliable. The campaign may not have changed at all; the estimate of it did. Early results deserve wide uncertainty, and judging them too soon can kill a campaign that was working.
The transcript.
For readers who prefer dialogue without zooming in.
Read the dialogue ↘
Marketer: The first six customers averaged $2,900. Our target is $4,000.
Dr. Bayes: So you killed the campaign?
Marketer: No. We waited.
Marketer: Now we're above target.
Marketer: So the campaign got better.
Dr. Bayes: Maybe. But what definitely got better was your estimate of the campaign.
Dr. Bayes: Six customers told you what happened first.
Dr. Bayes: A hundred started telling you what was actually happening.