Which statistical test should I use?
Almost every test choice is settled by three questions about your design — not by looking at your data. Answer them in order and the decision table below gives you the test.
Describe the outcome (continuous, ordinal, or categorical), count the groups or measurements, and say whether the same cases appear more than once. That is the whole decision. Assumption checks come after, and they decide between two versions of a test — not which family you are in.
Step 1 — what kind of outcome do you have?
The outcome is the thing you are trying to explain or compare. Its measurement level matters more than anything else in the choice.
| Outcome type | Examples | What it rules in |
|---|---|---|
| Continuous | Blood pressure, test score, reaction time, weight | Means: t tests, ANOVA, regression. Correlations with other continuous variables. |
| Ordinal | Likert items, pain graded 1–4, education level | Rank tests by default; treat as continuous only if the scale genuinely behaves that way. |
| Categorical | Diagnosis yes/no, group membership, colour chosen | Counts: chi-square, Fisher's exact test, logistic regression. |
Step 2 and 3 — how many groups, and is it the same people?
These two answers combine into a single cell. "Paired" means the same case contributes a value to more than one column — before and after, left and right, condition A and condition B.
| Design | Outcome | Use | If assumptions fail |
|---|---|---|---|
| 2 independent groups | Continuous | Independent-samples t test (Welch when variances differ) | Mann–Whitney U |
| 2 paired measurements | Continuous | Paired-samples t test | Wilcoxon signed-rank |
| 3+ independent groups | Continuous | One-way ANOVA | Kruskal–Wallis |
| 3+ repeated conditions | Continuous | Repeated-measures ANOVA | Friedman |
| Two categorical variables | Categorical | Chi-square or Fisher | Fisher when expected counts are small |
| Two continuous variables | Continuous | Pearson correlation | Spearman rank correlation |
| Continuous outcome, several predictors | Continuous | Multiple linear regression | Transform, or use robust standard errors |
| Binary outcome, several predictors | Categorical | Logistic regression | — |
The five comparisons people actually get wrong
These are the branches where a plausible-sounding choice is the wrong one. Each has its own page with a worked example computed from real numbers.
t test or ANOVA?
With two groups they are the same test: F equals t². The real reason ANOVA exists is what happens with three or more groups, where running repeated t tests inflates the false-positive rate to roughly 14% for three groups and 40% for five.
Do I need a nonparametric test?
"Not normal" is not sufficient reason. It depends on how far from normal the data are and how large the sample is — the t test relies on the sampling distribution of the mean, which settles down quickly.
Three measurements on the same people
Three separate paired t tests is the common wrong answer. Repeated-measures ANOVA exists because the three comparisons are not independent, and it also carries a sphericity assumption that has to be checked.
Chi-square or Fisher's exact?
The rule of thumb about expected counts below five is not folklore: in one worked example the chi-square p value sits exactly at .050 and Fisher's exact test gives .107.
Pearson or Spearman?
Not a full page yet, but the short version: one outlier moved a Pearson r from 1.000 to 0.198 while Spearman's ρ only fell to 0.538. If the relationship is monotone rather than linear, Spearman answers the question you actually asked.
Why the order matters
People often start from the data — "the histogram looks skewed, so I need a nonparametric test". That is the wrong end. The design fixes the family of test; the data only chooses between variants within it, or tells you the assumption a reviewer will ask about.
Starting from the design also protects you from a subtler error: choosing a test because it gives a smaller p value. Once you know the design admits one test and one fallback, there is nothing left to shop around with.
What to report either way
Report the test statistic with its degrees of freedom, the p value, the effect size, and — for comparisons of means — the mean difference with a 95% confidence interval. The effect size is not optional decoration: a p value says whether an effect is detectable at your sample size and nothing about its size.
When you would rather not choose
StatsToTable asks the three questions above and then selects the procedure, runs the assumption checks, and switches to the fallback when they fail — showing why. You can also read how each default was chosen.
t test vs ANOVA · Parametric vs nonparametric · Repeated measures · Chi-square vs Fisher · How the numbers are verified