Sampling and simulation
Watch probability and statistics happen: the central limit theorem from any distribution (even one you draw), the law of large numbers, confidence-interval coverage, A/B test planning and the cost of peeking, the bootstrap, permutation tests, Simpson's paradox, multiple comparisons, Monty Hall, two-dice events, balls into bins, coupons, inverse-transform and reservoir sampling — all on a seeded generator, so every picture can be linked.
Added
How to use it
Pick a tab. Every simulation runs on a seeded random-number generator: the same settings and the same seed always give the same picture, and New draws moves to the next seed. Under each tab, Link to exactly this gives a short URL that reopens what you see.
⛶ Full screen (or F) gives the open tab’s main plot the whole screen — the browser’s full
screen where it is allowed, otherwise a full-window layer (iPhone). The plot is redrawn as tall as
the screen allows, and the controls stay above it. Escape leaves.
-
Central limit — choose where one value comes from (a die, a coin, an exponential, the two-spike “0 or 100”, a uniform, the Cauchy, or a histogram you draw by hand by dragging over the left chart) and the sample size n. The tool draws thousands of samples, plots the histogram of their means against the predicted normal curve N(μ, σ²/n), and a tail meter gives the fraction beyond ±1.96 standard errors against the normal’s 5%. With the Cauchy the curve disappears: the spread of the means does not shrink however large n is.
-
Large numbers — running averages of several independent runs on a logarithmic axis, inside the funnel μ ± 1.96σ/√n.
-
Confidence intervals — “catch the mean”: each press adds an interval x̄ ± z*·s/√n (or t*) to a stack, blue if it covers the true mean 50 and red if it misses, with the running coverage. Tick biased sampling to draw from a population shifted by 4: the intervals narrow as n grows, and coverage collapses.
-
A/B planner — baseline rate, minimum detectable effect, α and power give the sample size per group; the plot shows the null and alternative distributions of z moving apart. Run A/B and Run A/A simulate an experiment day by day as a z-score random walk; tick peek to stop at the first crossing. The counter replays many A/A experiments: about 5% are significant at the end, but far more if you peek every day.
-
Bootstrap — resample the data with replacement (watch one resample at a time, stacked by repeats), build the distribution of the median, mean or a percentile, and read off the percentile interval. Paste your own numbers, or use the 200 stand-in response times.
-
Permutation test — shuffle the group labels, recompute Welch’s t each time, and compare the permutation p-value (simulated, and exact over every split when there are few enough) with the t-test’s.
-
Simpson — the kidney-stone data: slide how many small-stone patients each treatment got and watch the overall rates cross, while the rates within each group never change.
-
Multiple tests — m tests per experiment, some or none with a real effect: one experiment’s p-values against α, α/m and the Benjamini–Hochberg line, and the chance of at least one false positive over many experiments, with no correction, Bonferroni or BH.
-
Monty Hall — 3 to 100 doors, the host opening any number of them, and host knows versus host opens at random (games where a random host reveals the car are thrown away). Play one game slowly or thousands quickly; the exact answers are drawn as dashed lines.
-
Two dice — click cells to define events A and B (or pick them from the lists), see P(A), P(B), P(A ∩ B), P(A | B) and P(B | A) as exact fractions, whether A and B are independent, and condition on B fade every outcome outside B. Roll 1,000 times checks it by simulation.
-
Balls & bins — requests thrown at servers with one random choice or the best of two: the tallest stack, the idle fraction against (1 − 1/n)ⁿ ≈ 37%, and a histogram of loads.
-
Coupons — one collection drawn as distinct coupons against draws, the long wait for the last one, and the distribution of collection times against n·Hₙ.
-
Inverse transform — uniform heights u carried across to the CDF and down to x, for the exponential (x = −ln(1 − u)/λ) or a discrete distribution, with the histogram of many samples against the target.
-
Reservoir — Algorithm R step by step (keep item i with probability k/i), and the frequency with which each item ends in the sample over many runs: k/N for every item.
This tool simulates; to calculate with your own data use the Statistics tool (descriptive statistics, intervals, t-, z- and χ² tests, distribution tables) and Probability (exact dice distributions, counting, poker and lottery odds, the birthday problem, Bayes). Counting itself — Pascal’s triangle, stars and bars, derangements, birthday collisions — is in Counting, visually.
Companion to chapters 29–31 of Mathematics, Reconnected: Probability, Random variables and distributions and Statistics.
Limits
-
The generator is mulberry32, seeded from the settings and the seed. It is reproducible and good enough for teaching, not for cryptography. The book’s numbers were produced with Python’s and NumPy’s generators, so the simulated numbers here differ a little from the printed ones (the book gets a permutation p of 0.0067; this tool’s exact count over all 12,870 splits is 86/12,870 ≈ 0.0067, and its simulations land within sampling noise of it). The exact values — n·Hₙ, 2/3, the 14,748 users per group — match exactly.
-
The bootstrap’s stand-in data are 200 lognormal values drawn with this tool’s generator, not the book’s NumPy sample, so its median and p99 differ slightly from the printed 121.8 and 462.8.
-
A/B daily counts above 200 users per arm are drawn from the normal (or, for rare events, Poisson) approximation to the binomial. Intervals in Confidence intervals use z* by default, as the book’s lab does; choose t* for small samples.
-
Simulations run in the page; the largest settings (50,000 experiments of n = 400) take a moment.
Linking to an example
Links name the tab in the hash (#clt, #lln, #ci, #ab, #boot, #perm, #simpson, #multi,
#monty, #dice, #bins, #coupon, #inverse, #reservoir) and its settings as short
parameters, for example
?dist=exp&n=30&seed=3#clt or
?doors=4&open=2#monty. Parameters apply to the
tab in the hash; anything left out takes its default. ?example=<name> loads a book example (and
its tab, if the hash names none); other parameters then override it, so
?example=ch30-clt-exponential&n=5 is that example with n = 5. Values are plain numbers or
words, so they need no escaping, except lists: separate them with commas (a=212,198,225).
Opening a link replaces this browser’s saved settings for the tool; Restore my settings undoes it.
| Tab | Parameter | Meaning | Example |
|---|---|---|---|
all |
|
the random seed (integer); same seed, same picture |
|
|
|
source: |
|
|
|
sample size, 1–400 |
|
|
|
number of sample means, 100–50,000 |
|
|
|
P(heads) for |
|
|
|
hand-drawn histogram on [0, 1]: 2–60 bar heights, commas |
|
|
|
source (as above, except |
|
|
|
population: |
|
|
|
sample size, confidence level in %, |
|
|
|
shift of the population sampled from (the book’s switch is 4); 0 is unbiased |
|
|
|
number of intervals drawn |
|
|
|
baseline rate and minimum detectable effect, as fractions (0.01 = 1 point) |
|
|
|
significance level and power |
|
|
|
users actually run, as a multiple of the required number |
|
|
|
number of daily looks; |
|
|
|
number of A/A replays for the false-positive counter |
|
|
|
the sample, numbers separated by commas (blank: the stand-in latencies) |
|
|
|
|
|
|
|
the two groups (commas) and the number of shuffles |
|
|
|
small-stone patients (of 350) given treatment A and B |
|
|
|
tests per experiment, α, |
|
|
|
how many tests have a real effect; number of experiments |
|
|
|
doors (3–100), doors the host opens (0: all but one), |
|
|
|
games played |
|
|
|
events: |
|
|
|
|
|
|
|
servers, requests (0: one per server), |
|
|
|
coupon types; runs for the histogram |
|
|
|
|
|
|
|
stream length, reservoir size, runs |
|
Book examples (?example=):
| Example | Shows | Book |
|---|---|---|
P(total 8 | red die even) = 3/18 = 1/6, the book’s conditioning example |
ch. 29 |
|
A = total 8, D = doubles: A ∩ D = {(4,4)}, A ∪ D has 10 outcomes |
ch. 29 |
|
“total is 7” is independent of “red die is even” |
ch. 29 |
|
Exercise: P(at least one six | total ≥ 10) = 5/6 |
ch. 29 |
|
host knows: switching wins 2/3 |
ch. 29 |
|
host opens a door at random: 1/2 and 1/2 |
ch. 29 |
|
Exercise: 4 doors, host opens 2 goats, switching wins 3/4 |
ch. 29 |
|
100 doors, 98 goats opened: switching wins 99/100 |
ch. 29 |
|
three runs of 10 000 die rolls pinned ever closer to 3.5 |
ch. 30 |
|
the proportion of heads converges to p |
ch. 30 |
|
Cauchy running means never settle |
ch. 30 |
|
means of 30 exponential(1) values: SD 1/√30, tails 1.4% / 3.3% |
ch. 30 |
|
n = 1: the skewed source itself |
ch. 30 |
|
means of 5 die rolls |
ch. 30 |
|
the “0 or 100” distribution, n = 30 |
ch. 30 |
|
a hand-drawn two-humped source |
ch. 30 |
|
Cauchy: averages of 100 are as wild as one value |
ch. 30 |
|
50 coupons: n·H_n ≈ 225 draws |
ch. 30 |
|
one random choice: about 37% idle |
ch. 30 |
|
best of two choices: a far lower tallest stack |
ch. 30 |
|
x = −ln(1 − u)/0.5: mean ≈ 2, P(x > 3) ≈ 0.223 |
ch. 30 |
|
walking up the CDF staircase for a, b, c |
ch. 30 |
|
k = 3 from a stream of 10: every item kept with probability 0.3 |
ch. 30 |
|
fifty 95% intervals from samples of 25 |
ch. 31 |
|
biased sampling: narrow intervals that miss |
ch. 31 |
|
skewed population, n = 5, z*: coverage falls short |
ch. 31 |
|
10% → 11% at α 0.05, power 80%: about 14 750 per group |
ch. 31 |
|
a 0.5-point lift needs about four times as many |
ch. 31 |
|
15 looks at 1 000 users per arm: false positives ≈ 23% |
ch. 31 |
|
bootstrap CI for the median of 200 response times |
ch. 31 |
|
the p99 interval: wide and lopsided |
ch. 31 |
|
old vs new build: t = 3.23, permutation p ≈ 0.007 |
ch. 31 |
|
kidney stones: A wins in each group, B overall |
ch. 31 |
|
same case mix for both: the paradox vanishes |
ch. 31 |
|
20 tests of true nulls: 64% find “something” |
ch. 31 |
|
Bonferroni: any false positive ≤ 5% |
ch. 31 |
|
Benjamini–Hochberg on 100 tests, 10 real effects |
ch. 31 |
Feedback is a gift. Something to improve, a feature to add, or a bug? Tell me.
Runs entirely in your browser; nothing you type leaves this page.