Snippets about: Statistics
Scroll left and right !
Pareto Principle And The Vital Few
The Pareto Principle states that 80% of results come from 20% of efforts. A few key activities contribute most of the value in any given situation:
- In business, 80% of profits come from 20% of customers
- In society, 20% of criminals commit 80% of crimes
- In personal lives, 20% of activities produce 80% of satisfaction
Essentialists invest their time and energy only in the vital few choices and activities that matter most, with the understanding that most things are unimportant distractions.
Section: 1, Chapter: 3
Book: Essentialism
Author: Greg McKeown
Bayes' Theorem: Updating Beliefs with Evidence
Bayes' Theorem, developed by Thomas Bayes, is a fundamental principle for updating probabilities based on new evidence.
The theorem is expressed as P(A|B) = P(B|A) * P(A) / P(B), where P(A|B) is the probability of A given B, P(B|A) is the probability of B given A, P(A) is the prior probability of A, and P(B) is the total probability of B.
This formula allows us to calculate how the probability of an event changes based on new information. Bayes' Theorem is crucial in many real-world scenarios, from medical diagnosis to legal reasoning. For example, it explains why a positive result on a medical test doesn't necessarily mean a high probability of having the disease, especially if the disease is rare in the population. Understanding and applying Bayes' Theorem can lead to more accurate interpretations of probabilistic information in various fields.
Section: 1, Chapter: 6
Book: The Drunkard's Walk
Author: Leonard Mlodinow
Regression To The Mean
The phenomenon of regression to the mean explains why we often misinterpret the effects of rewards and punishments. When we observe an exceptional performance, which is likely due to a combination of skill and luck, we tend to reward the individual. However, on subsequent attempts, their performance is likely to regress back towards their average level, making it appear as if the reward has backfired. Conversely, when we observe an unusually poor performance, we tend to punish the individual, and their subsequent improvement due to regression to the mean makes it seem like the punishment was effective.
Section: 2, Chapter: 17
Book: Thinking, Fast and Slow
Author: Daniel Kahneman
Why Base Rates Matter
Base rates refer to the underlying probability of an event or category. For example, if 1% of the population has a particular disease, then the base rate of having the disease is 1%. When making predictions or judgements about individual cases, we often neglect base rates and instead focus on specific information about the case, even if that information is unreliable or uninformative. This can lead to significant errors, as we fail to consider the underlying probability of the event.
Section: 2, Chapter: 14
Book: Thinking, Fast and Slow
Author: Daniel Kahneman
Using Statistics To Place Activities On The Continuum
Chapter 3 presents a method for empirically placing activities on the skill-luck continuum, using three statistics:
- Demonstrated consistency of outcomes across repeated trials (e.g. year-to-year correlation of batting averages)
- The distribution of success among the population of participants (e.g. number of players hitting above .300)
- The ability of a measure to predict future performance (e.g. correlation of on-base percentage to future runs scored) Using these measures, we can estimate the relative contribution of skill and luck in different domains.
Section: 1, Chapter: 3
Book: The Success Equation
Author: Michael Mauboussin
Insights for Quantifying Uncertainty
Actionable insights for AI developers:
- Make AI systems' confidence scores actually reflect statistical uncertainty, not just relative ranking
- Build pipelines for "uncertainty handoff" to human oversight in high-stakes applications
- Extensively test AI systems on out-of-distribution and adversarial inputs to probe overconfidence
- Favor objectives and learning procedures that are robust to uncertainty over brittle "point estimates"
The upshot is that well-calibrated uncertainty is a feature, not a bug, for AI systems operating in the open world. We should invest heavily in uncertainty estimation techniques and make them a core component of AI system design.
Section: 3, Chapter: 9
Book: The Alignment Problem
Author: Brian Christian
The Power Laws Of Performance
Mauboussin examines the surprising statistical regularities that show up across many domains of human performance. The key finding is that many performance metrics follow a power law distribution, wherein:
- A small number of top performers account for a disproportionate share of the total output
- The gap between the best and the rest is much larger than a normal distribution would predict.
For Example:
- City sizes (a few mega-cities, many small towns)
- Wealth distribution (a few billionaires, many people of modest means)
- Bestseller lists (a few blockbuster hits, many low-selling titles)
- Scientific citations (a few papers with massive impact, many rarely-cited papers)
The power law pattern arises from the complex social dynamics that shape success in these domains - in particular, the rich-get-richer effects of cumulative advantage.
Section: 1, Chapter: 6
Book: The Success Equation
Author: Michael Mauboussin
How Many Trucks Does It Take?
During World War II, the Allies wanted to estimate the number of German tanks being produced each month. Intelligence officers simply looked at serial numbers printed on captured German tanks. If tank #1,242 was built in May and tank #1,867 was built in June, then they assumed 625 tanks were built in June.
After the war, data from German factories showed that the traditional intelligence estimates were off by a factor of 2.5. But the serial number analysis was off by only 1%.
Imagine a jar filled with marbles labeled from 1 to N. You draw a random sample of marbles, noting their numbers. How can you estimate the highest number N - the total number of marbles in the jar?
Statisticians showed that the best estimator is to take the highest drawn number and multiply it by (k+1)/k, where k is the number of draws. So if you drew 4 marbles with a highest number of 78, you'd estimate there were 97.5 marbles total.
The key is that missing marbles provide information, since they indicate the jar contains numbers you haven't seen yet. The estimator performs optimally by considering both what was drawn and what likely wasn't drawn yet, given the sample. A Bayesian approach outperforms one using the observed data alone.
Section: 1, Chapter: 6
Book: Algorithms to Live By
Author: Brian Christian
Surprising Virtue Of Persistence And Ignorance
Suppose you arrive in a new city and see a long line outside a restaurant. Knowing nothing else, what does that tell you about the restaurant's quality?
According to Bayes's Rule, long lines are more likely to form for good restaurants than bad ones. Even with no other information, the line is evidence of quality. The longer the line, the better the restaurant is likely to be.
But now imagine you go into the restaurant and have a terrible meal. What should you believe now - your own experience or the line of people waiting? Again, Bayes's Rule says to weigh the evidence based on sample size. A single bad meal is a small sample. The long line reflects dozens of people's opinions. It should still dominate your judgment.
The same idea applies to product reviews, survey results, or any wisdom of the crowd. If your own experience contradicts the majority, trust the majority. Your own sample size is too small to outweigh the group. First impressions can lead you astray.
Section: 1, Chapter: 6
Book: Algorithms to Live By
Author: Brian Christian
The Three Measures Of A Useful Statistic
A good metric should have three key properties:
- Consistency - Does the metric reliably measure the same thing across time and contexts?
- Predictive power - Does the metric actually predict the outcome we care about?
- Noisiness - How much random variability is there in the metric relative to the signal?
The best metrics are consistent, predictive, and have a high signal-to-noise ratio. Examples of useful metrics include:
- On-base percentage in baseball (consistent, predictive of scoring, less noisy than batting average)
- Customer retention rate in business (consistent, predictive of profits, less noisy than raw sales numbers)
- Sharpe ratio in investing (consistent way to measure risk-adjusted returns, predictive of fund quality)
Section: 1, Chapter: 7
Book: The Success Equation
Author: Michael Mauboussin
Kelly Criterion Betting Demonstrates How Practical Risk Management Diverges From Naive Expected Value Maximization
The Kelly criterion is a formula for optimal bet sizing that maximizes long-run returns while minimizing risk of ruin. A simulation of betting on 272 NFL games in a season, using either full Kelly sizing or 5x Kelly, illustrates the tradeoffs:
- Full Kelly generates a healthy 24% average return while having only a 10% chance of losing money and a 2% chance of going broke.
- 5x Kelly produces much higher average returns, but goes broke 88% of the time. It's higher EV but with an unacceptable risk of ruin.
The exercise shows how practical risk takers need to balance expected value with sustainability. Overbetting relative to Kelly, as SBF was prone to, is a recipe for disaster. Real risk management requires resilience, not just maximizing some theoretical long-run average.
Section: 2, Chapter: 8
Book: On The Edge
Author: Nate Silver
Reversion To The Mean: The Cruel Equalizer
Chapter 1 also introduces the concept of reversion to the mean - the idea that an outcome that is far from the average will be followed by an outcome that is closer to the average. Reversion to the mean occurs whenever two measures are imperfectly correlated. The lower the correlation, the more extreme the reversion. In activities with a significant element of luck, reversion to the mean can be a powerful force. A few key implications:
- Don't overreact to extreme performances, good or bad. They are likely to revert toward the average.
- Be wary of paying for past performance. The "hot hand" may be nothing more than luck.
- Anticipate reversion in your own performance. If you have a great year, don't expect to maintain that level. If you have a terrible year, better times are likely ahead.
Section: 1, Chapter: 1
Book: The Success Equation
Author: Michael Mauboussin
Statistical vs Causal Base Rates
There are two types of base rate information, and they are treated differently by our minds:
Statistical Base Rates: These are facts about the overall population or category, such as the percentage of people who have a college degree. We tend to underweight or ignore statistical base rates when specific information about an individual case is available.
Causal Base Rates: These are facts that have a direct causal bearing on the individual case. For example, if you learn that a student attends a highly selective university, this information has a causal influence on your judgement of the student's academic abilities. Causal base rates are treated as information about the individual case and are readily combined with other case-specific information
Section: 2, Chapter: 16
Book: Thinking, Fast and Slow
Author: Daniel Kahneman
The Concept of Mathematical Expectation
Pascal introduced the concept of mathematical expectation, which calculates the average outcome of a random process if it were repeated many times. This idea is crucial in decision theory and game theory. Pascal applied it to his famous "wager" about belief in God, arguing that the potential infinite gain of believing outweighs any finite cost. In modern times, expectation is used in various fields, from insurance to gambling, to quantify the likely outcome of uncertain events. Understanding expectation can help in making rational decisions under uncertainty.
Section: 1, Chapter: 4
Book: The Drunkard's Walk
Author: Leonard Mlodinow
Galton's Contributions to Statistics
Francis Galton, a cousin of Charles Darwin, made significant contributions to the field of statistics. His key contributions include:
- Regression to the mean: The observation that extreme characteristics in parents tend to be less extreme in their children
- Correlation: A measure of the strength of association between two variables
- The concept of standard deviation as a measure of variability
Galton's work laid the foundation for many modern statistical techniques and highlighted the importance of understanding variation in populations. His ideas about regression to the mean, in particular, have important implications for understanding phenomena in fields ranging from genetics to sports performance.
Section: 1, Chapter: 8
Book: The Drunkard's Walk
Author: Leonard Mlodinow
The High Cost Of Selective Attention
Imagine you're considering two slot machines to play at a casino. For machine A, you watch a dozen people play and 4 of them win. For machine B, you have no information. Which should you choose?
Most people avoid uncertainty and pick machine A. But rationally, machine B is more likely to pay off. Even though you have no data, the chances it pays off more than 33% are better than the chances it pays off less.
This is a direct implication of Laplace's Law: if you have no prior information, the probability of a positive outcome is (k+1)/(n+2) where k is the number of positive outcomes observed out of n attempts. For machine A, that's (4+1)/(12+2) = 36%. For machine B it's (0+1)/(0+2) = 50%.
Human attention, and media coverage, is drawn to the vivid and available, not the representative. This selective attention means that more media coverage of an event indicates that it's more rare, not necessarily more common. To be a good Bayesian, look for the silent evidence as much as the headline grabbing stories. Often what you haven't seen is as informative as what you have.
Section: 1, Chapter: 6
Book: Algorithms to Live By
Author: Brian Christian
Sports on the Continuum
- Basketball statistics show high year-to-year correlations, a dispersed distribution of success, and strong predictive power - indicating a strong role for skill
- Hockey statistics show lower consistency, a more concentrated success distribution, and less predictive power - indicating a larger role for luck These statistical tools give us an objective way to assess the skill-luck balance in different domains and make better predictions.
Section: 1, Chapter: 3
Book: The Success Equation
Author: Michael Mauboussin
The Law of Large Numbers
The law of large numbers, formalized by Jakob Bernoulli, states that as the number of trials of a random process increases, the average of the results will tend to converge on the expected value.
This principle is crucial for understanding how probability manifests in the real world. For example, while a small number of coin flips might deviate significantly from 50% heads, a large number of flips will tend very close to this expected ratio. This law underpins many statistical methods and helps explain why large samples are generally more reliable than small ones.
Section: 1, Chapter: 3
Book: The Drunkard's Walk
Author: Leonard Mlodinow
The Normal Distribution: Nature's Favorite Pattern
The normal distribution, also known as the bell curve, is a fundamental concept in statistics. It describes a symmetrical, bell-shaped distribution of data where most observations cluster around the mean, with fewer observations towards the extremes. Key properties:
- About 68% of observations fall within one standard deviation of the mean
- About 95% fall within two standard deviations
- About 99.7% fall within three standard deviations
This distribution is ubiquitous in nature and human affairs, from human height to measurement errors in scientific experiments. Understanding the normal distribution is crucial for interpreting data and making inferences about populations from samples.
Section: 1, Chapter: 7
Book: The Drunkard's Walk
Author: Leonard Mlodinow
Blending Skill And Luck To Match Real-World Results
Mauboussin illustrates the skill-luck continuum using the example of the NBA draft lottery. In the lottery, the worst teams get the highest probability of winning the top draft picks, in an effort to maintain competitive balance.
Mauboussin and his colleagues simulated the draft lottery under two scenarios: one with pure luck (a weighted coin flip) and one with pure skill (the worst team always gets the #1 pick). They then compared the actual results to these two extremes.
The results showed that the real NBA draft lottery results fell almost exactly halfway between pure luck and pure skill, suggesting that the lottery system does a good job of blending luck and skill to achieve its intended purpose. This type of simulation - comparing real-world results to luck and skill extremes - can be a useful tool for placing activities on the skill-luck continuum.
Section: 1, Chapter: 3
Book: The Success Equation
Author: Michael Mauboussin
The Peril of P-Hacking in Social Science Research
P-hacking, or manipulating data analysis to achieve statistically significant results, is a major problem in academic research. It can lead to the publication of spurious findings and distort our understanding of the world. Key issues:
- Researchers often face strong incentives to "torture the data" until they get p-values under 0.05 (the usual publication threshold).
- Analytical flexibility allows many "forking paths" to significant results, even if the underlying effect is negligible or nonexistent.
- Publication bias means p-hacked studies get published while negative results languish in file drawers, skewing the record.
Solutions like pre-registration of analysis plans and emphasizing effect sizes over statistical significance are gaining traction. But the deeper issue is a flawed model of "one-shot" science that fetishizes single, surprising results over repeated, incremental progress.
Section: 1, Chapter: 11
Book: Fluke
Author: Brian Klaas
