Calculator Inputs
Pick a test, enter summary statistics, then calculate significance. Results appear above this form after submission.
Example Data Table
| Scenario | Suggested test | Example inputs | What the result tells you |
|---|---|---|---|
| Average delivery time versus target | One-sample t-test | n = 24, mean = 82.4, SD = 7.5, μ₀ = 80 | Checks whether the sample mean differs from the benchmark. |
| Known process variance quality check | One-sample z-test | n = 50, mean = 12.4, σ = 2.8, μ₀ = 11.5 | Useful when population deviation is known and stable. |
| A/B metric comparison | Two-sample Welch t-test | Group A: 35, 18.7, 4.9; Group B: 32, 16.1, 5.4 | Compares independent means without assuming equal variances. |
| Before versus after program change | Paired t-test | n = 18, mean diff = 2.2, SD diff = 3.1 | Measures whether average within-pair change is meaningful. |
| Conversion rate against prior expectation | One-proportion z-test | n = 200, successes = 118, p₀ = 0.50 | Shows whether the observed rate differs from the hypothesized rate. |
Formulas Used
p-value logic: the calculator converts the test statistic into a probability under the null distribution. A smaller p-value means the observed result would be rarer if the null hypothesis were true.
Decision rule: reject the null hypothesis when the p-value is less than or equal to α. Otherwise, fail to reject the null hypothesis.
Confidence interval: the interval is built with the corresponding critical value and standard error. If the null reference value falls outside a two-sided interval, the two-sided test is significant at the same alpha.
Effect size: Cohen’s d and dz summarize mean differences relative to variation. Cohen’s h summarizes proportion differences on an arcsine scale. Statistical significance and practical importance are not the same thing.
How to Use This Calculator
- Select the correct hypothesis test for your data structure.
- Choose a two-tailed, left-tailed, or right-tailed alternative.
- Enter α, usually 0.05 unless your study requires another threshold.
- Fill in the summary statistics requested for the chosen test.
- Submit the form to see the result block above the calculator.
- Review the test statistic, p-value, interval, effect size, and decision.
- Use the graph to inspect where the statistic falls versus critical cutoffs.
- Download the result as CSV or PDF for reporting or review.
Frequently Asked Questions
1) In formulating hypotheses for a statistical test of significance, the null hypothesis is often what?
It is usually a statement of no effect, no difference, or no association. The null provides the benchmark distribution used to compute the test statistic and p-value.
2) In hypothesis testing, is the test statistic ever called significant?
Not really. The result of the test is called statistically significant, not the raw statistic itself. The statistic is the numerical input used to obtain the p-value or compare with a critical value.
3) If the results of a significance test are statistically significant at α = 0.05, then what follows?
It means the p-value is 0.05 or smaller, so you reject the null hypothesis at that level. It does not prove the null is false, and it does not guarantee practical importance.
4) How to test for statistical significance?
State null and alternative hypotheses, choose α, compute the correct test statistic, obtain the p-value from the matching distribution, then compare p-value with α and report the conclusion with context.
5) Which of the following is true of statistical significance testing?
A true statement is that significance testing evaluates evidence against the null hypothesis using sample data and a probability model. It does not directly measure the size or usefulness of an effect.
6) Does a small p-value always mean the effect is important?
No. A tiny effect can become significant with a very large sample. Always review effect size, interval width, data quality, and subject-matter impact alongside the p-value.
7) When should I use a paired t-test instead of a two-sample test?
Use a paired t-test when each observation in one condition is naturally matched with one in another condition, such as before-and-after measurements on the same subjects.
8) What assumptions matter before interpreting significance?
Check independence, sensible measurement quality, reasonable distribution assumptions, and whether the test choice fits your design. For mean tests, extreme outliers and heavy skew can distort conclusions.