Starting from zero · Course 10 of 11 · Stage 2
Statistics and evidence
Standard errors, p-values, multiple testing, survivorship and overfitting, worked on real US market cohorts. How to tell a finding from a fitted result.
This industry sells claims about what works. Almost all of them are supported by a number, and most of those numbers are weaker than they look. This course is how to tell which is which.
Nine lessons, in order. The first three build the machinery: what a sample can support, how to measure spread, and how far a measured average can be from the truth. The fourth states what a p-value is and the three things it is routinely taken to mean and does not. The fifth is the one that matters most: what happens to significance when many rules are tried and the best is reported.
The last four are the failure modes. Correlation without cause. A fitted line that explains one percent of anything. A sample that deleted its own failures. And a model with more freedom than data, which is what most published backtests are.
Every statistic here is computed on this product's own US market data, with the cohort size and the reading date beside it. Two of them produce uncomfortable results about figures this product publishes, and both are printed anyway. That is the standard the course is arguing for.
The tools, not the work
Statistics and evidence is 9 lessons on the arithmetic and the rules an investor has to be able to handle: the discounting, the probability, the standards, the tax. They are what you work WITH. None of them is work done on a company you own, and a tool understood is not a book checked.
Related
- Why backtests mislead is the applied version of this whole course, on the one question it exists to answer.
- Base rates first is the course before this one, and the counting discipline this one measures the uncertainty of.
- Benchmarks publishes the cohorts this course computes its standard errors and regressions on, with every size and build date stated.
See our methodology for how the product reads a filing and what it refuses to compute.