Why is it interesting?

For many extremely illuminating visualizations, see http://students.brown.edu/seeing-theory/

• Probability is starting with an animal, and figuring out what footprints it will make
• Statistics is seeing a footprint, and guessing the animal.

Probability is straightforward: you have the bear. Measure the foot size, the leg length, and you can deduce the footprints. "Oh, Mr. Bubbles weighs 400lbs and has 3-foot legs, and will make tracks like this." More academically: "We have a fair coin. After 10 flips, here are the possible outcomes."

Statistics is harder. We measure the footprints and have to guess what animal it could be. A bear? A human? If we get 6 heads and 4 tails, what're the chances of a fair coin?#

A Brief Introduction to Probability & Statistics by Kalid Azad

Important related concepts:

• Bayesian Statistics

Layman

Student

Recommended Reading:

• A Brief Introduction to Probability & Statistics by Kalid Azad
• Introduction to the concept of likelihood and its applications by Alexander Etz
• What is Statistical Significance? by Tommaso Dorigo
• http://www.science20.com/tommaso_dorigo/decision_trees_explained_to_kids-224948
• Stats without Tears by Stan Brown (free online textbook)
• Statistics: (for Bayesian) Statistical Rethinking - A Bayesian Course with Examples in R and Stan - by McElreath
• DISCOVERING THE SIGNIFICANCE OF 5σ Louis Lyons

Researcher

• Data Analysis in High Energy Physics: A Practical Guide to Statistical Methods by Olaf Behnke et. al.
• Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations by Sander Greenland et. al.

• Probability Theory as Extended Logic Edwin T. Jaynes - Washington University website
• Probability Theory: The Logic of Science by Edwin T. Jaynes

Examples

Example1

Example2:

History