p-value Calculator

About p-value Calculator

How the p-value Is Calculated

The p-value is the tail-area of the reference sampling distribution beyond the observed statistic. For a two-tailed test it is 2 × min(left-tail, right-tail); for a one-tailed test it is whichever single tail the alternative hypothesis specifies. The formulas used here follow the definitions in the NIST/SEMATECH Engineering Statistics Handbook.

Worked example (two-tailed z-test). A coin is flipped 100 times and lands heads 62 times. Under the null hypothesis p = 0.5, the expected count is 50 with standard deviation √(100 × 0.5 × 0.5) = 5. The test statistic is z = (62 - 50) / 5 = 2.40. The right-tail area for z = 2.40 under the standard normal is 0.00820; doubling gives a two-tailed p-value of 0.01640. Because 0.0164 < 0.05, the null hypothesis is rejected at the 5% level but not at the 1% level (since 0.0164 > 0.01).

Under the hood the calculator uses the Abramowitz-Stegun 5-term approximation for the normal CDF (accurate to about 7 decimal places) and the regularised incomplete beta and gamma functions via continued fractions for the t, chi-square, and F distributions. These are the same numerical recipes used by R's pt(), pchisq(), and pf() internals.

Interpreting p-values at Different Alpha Levels

Smaller p-values indicate data that would be more surprising under the null hypothesis. The five-band convention below originated with Fisher (1925) and is the labelling used in most scientific journals today.

Different fields adopt stricter or looser thresholds. Particle physics requires 5σ (p ≈ 2.87 × 10^-7) to claim discovery - the standard used for the 2012 Higgs boson announcement at CERN. Genome-wide association studies typically use 5 × 10^-8 to control for multiple testing across ~1 million independent loci. Clinical trials often pre-register at 0.025 (one-sided) per FDA statistical guidance. A high-profile 2018 proposal by Benjamin and 71 co-authors in Nature Human Behaviour argued that new discoveries in social and biomedical sciences should be held to 0.005 rather than 0.05.

Which Test Statistic Should I Use?

The test statistic is chosen by what is known about the population and what kind of data is being analysed. The four distributions covered here account for the overwhelming majority of classical frequentist tests in introductory and applied statistics.

As the t-distribution's degrees of freedom grow it converges to the standard normal - the two agree to three decimal places once df exceeds about 30, which is where the heuristic "n > 30 => use z" comes from. A common beginner mistake is using z with a small sample because the population standard deviation feels "known"; when σ is actually the sample SD, the sampling distribution has heavier tails and the t-distribution gives the correct p-value. For proportions, continuity correction (Yates) is usually not needed for n > 30 per cell but matters in small 2x2 tables.

One-Tailed vs Two-Tailed Tests

Choose one-tailed only when the scientific question is genuinely directional and an effect in the opposite direction would be treated the same as no effect. Most published research uses two-tailed tests because the cost of missing an unexpected reversal is usually higher than the gain in power.

Important. The direction must be declared before seeing the data. Switching from two-tailed to one-tailed after looking at the results is a form of p-hacking and doubles the real Type I error rate. FDA guidance for confirmatory clinical trials specifies that one-sided testing at 0.025 is equivalent in false-positive terms to two-sided at 0.05.

Common p-value Misconceptions

The American Statistical Association's 2016 statement on p-values (Wasserstein & Lazar, The American Statistician) listed six principles after polling dozens of statisticians. Three of the most persistent errors it called out are summarised below alongside the correct interpretation.

Type I and Type II Errors

Every hypothesis test has four possible outcomes depending on the truth of the null hypothesis and the decision made. The significance level α controls Type I errors; the probability of a Type II error is denoted β, and statistical power is 1 - β.

Cohen (1988) proposed 0.80 as a reasonable minimum for power in social science research, meaning researchers accept a 20% chance of missing a real effect of the size they care about. Medical trials increasingly target 0.90 power. The sample size calculator inverts the relationship to compute how many observations are needed for a target power given the effect size and alpha.

Example p-values for Common Test Statistics

These reference values can be used to sanity-check calculator output and to memorise the critical statistics that cross the 0.05 and 0.01 boundaries.

Reporting p-values in Practice

The APA Publication Manual (7th edition) recommends reporting exact p-values to two or three decimal places (e.g. p = 0.023) rather than just p < 0.05, except when the p-value is extremely small, in which case p < 0.001 is preferred. Never report p = 0.000 - that is a rounding artefact, not a true zero probability. For test statistics, include the degrees of freedom: t(24) = 2.31, p = 0.030 or F(2, 47) = 5.12, p = 0.010. Report effect size (Cohen's d, η², r) alongside every p-value so readers can assess practical significance.

For computing a z-score from a raw observation before feeding it into this tool, the z-score calculator handles standardisation. To turn a p-value into a confidence interval around the estimate, see the confidence interval calculator; the two are duality of the same Neyman-Pearson framework.

Limitations of Null Hypothesis Significance Testing

The p-value depends on the sample size as well as the effect. A correlation of r = 0.05 is practically meaningless but becomes significant at p < 0.05 once n exceeds about 1,550 - and at p < 0.001 past n ≈ 4,500. This is why modern best practice (ASA 2016, Nature 2019 manifesto signed by over 800 scientists) is to report p-values alongside confidence intervals and pre-specified effect-size thresholds rather than using p < 0.05 as a binary gatekeeper. Bayesian alternatives (Bayes factors, posterior probabilities) are gaining ground in fields like cognitive science and ecology but require prior specification that classical frequentist testing deliberately avoids.

All calculations run in your browser. No data is sent to any server.

Sources

• NIST/SEMATECH e-Handbook of Statistical Methods - Significance Levels
• American Statistical Association - Statement on p-Values (Wasserstein & Lazar, 2016)
• Benjamin et al. (2018) - Redefine Statistical Significance, Nature Human Behaviour
• Amrhein, Greenland & McShane (2019) - Retire Statistical Significance, Nature
• FDA - E9 Statistical Principles for Clinical Trials (ICH Guidance)
• APA Publication Manual - Reporting Numbers and Statistics
• CERN - What Five Sigma Means in Particle Physics