Lesson 3 of 7
Base rates and the outside view
Start a forecast from how often similar companies achieved it before adjusting for what makes this one different.
By Javier Sanz
Founder of ValueScreener. Built Ninety Nine, a securities services provider, and sold it to a tier-one global bank; ran operations and internationalization at Alpaca, the brokerage API.
Who writes thisEditorial processLinkedIn (opens in a new tab)X (opens in a new tab)
In one minute
- The outside view asks how often similar companies achieved something before asking what is special about this one.
- A test that is right 90 percent of the time can still be wrong about half of what it flags, if what it looks for is rare.
- Start every forecast from the outside view, then adjust for the specific evidence in front of you.
Before forecasting what a company will do, ask how often companies like it have done it. That question, the outside view, sets a starting point a forecast has to move away from with evidence, not with a feeling that this case is different.
100 companies, 10 real cases, a test that is right 90 percent of the time
- Real cases, correctly flagged9
- Real cases, missed1
- Not a real case, flagged anyway9
- Not a real case, correctly cleared81
The inside view feels more informed
The inside view builds a forecast from this company's own plan, this management's own targets, this quarter's own momentum. It feels more informed because it uses more specific detail. Kahneman and Lovallo argued that this is exactly why it fails. Forecasters anchor on the plan in front of them and discount the many similar plans that did not work out.
Why a good test still misleads
The figure above is not about one company. It is what happens whenever a test looks for something rare. Sensitivity of 90 percent means the test catches 9 of 10 real cases. Specificity of 90 percent means it correctly clears 81 of the 90 companies that are not real cases. But it still flags 9 of those 90 by mistake, and 9 false flags beside 9 true ones is a coin flip on any single flag.
Rarity matters as much as accuracy
Before the test, any one company had a 10 percent chance of being a real case. After a flag, it has 50 percent. The flag moved the odds fivefold; it did not make the case certain. A sharper test helps, but short of perfect, the rarer the thing, the larger the share of false flags. The 10 percent is the base rate: start there, then let the evidence move you.
A cohort is a base rate
A distribution built from many companies is the outside view already done for you. Mauboussin, Callahan and Majd's Base Rate Book is the outside view as a reference table. It shows how often companies of a given size achieved a given rate of sales growth, and similar outcomes. Before judging whether a company's margin, growth or return looks achievable, see how common that level is in its cohort today. The company's own plan is the inside view; the cohort is the outside view it has to be checked against.
Check yourself
A test correctly flags 90 percent of real cases and correctly clears 90 percent of non-cases. Out of 100 companies with 10 real cases, about what share of the flags are false?
- About 90 percent
- About 10 percent
- About 50 percent
Show the answer
Answer: C.
Nine real cases and nine false ones are flagged, eighteen in total, so about half of the flags are wrong even though the test is right 90 percent of the time on each group.What does the outside view start a forecast from?
- This company's own plan and targets
- How often similar companies achieved the same thing
- The forecaster's confidence in the analysis
Show the answer
Answer: B.
The outside view is the base rate: how often companies like this one actually achieved the outcome, before any adjustment for what seems special here.Why does a good test still produce many false flags when what it looks for is rare?
- Because rare events cannot be tested for
- Because the test is inaccurate
- There are far more non-cases than cases
Show the answer
Answer: C.
With 90 non-cases and only 10 cases, a 10 percent error rate on the much larger non-case group produces as many false flags as there are true ones.Sources
- Kahneman and Lovallo, 'Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking', Management Science, 1993, on the inside view and the outside view.
- Mauboussin, Callahan and Majd, 'The Base Rate Book: Integrating the Past to Better Anticipate the Future', Credit Suisse, 2016.
What this leaves out
- The 90 percent figures in the worked example are round and illustrative; a real test's sensitivity and specificity must be measured on its own record.
- A base rate is a starting point, not a verdict on one company. Specific evidence can and should move a forecast away from it.
- A base rate depends on the reference class you choose, and a cohort of today's listed companies leaves out the ones that failed, so it flatters success.
- Nothing here is advice about any security.