· Xiaojing Yang · Statistics · 2 min read
中文Correlation vs. Causation in AI Research
Correlation is useful evidence, but causal claims require stronger design and stronger assumptions.
Core idea
Correlation can guide investigation; causation requires a design that rules out alternative explanations.
1. The distinction
Correlation means two variables move together. Causation means changing one variable would change the other, under a defined intervention. AI research often slides from the first to the second too quickly.
X and Y move together
X helps predict Y
Changing X changes Y
We understand why
It holds beyond this setting
2. Why AI papers are vulnerable
Models are trained on messy data. Confounders are everywhere: domain, language, label quality, prompt style, annotator behavior, dataset source, and compute budget. A pattern may be real but not causal.
Correlation claim
Longer prompts are associated with better scores.
Causal claim
Making prompts longer improves scores, all else controlled.
3. AI/NLP example
Suppose a multilingual model performs worse on one language. The language itself may not be the cause. The real causes may include lower training data quality, different domain mix, tokenization inefficiency, or poorer evaluation data.
4. Better habits
| Habit | Why it helps |
|---|---|
| Name confounders | Prevents simplistic explanations |
| Use controlled comparisons | Makes alternatives less plausible |
| Prefer ablations | Tests mechanism |
| Avoid causal language when design is observational | Keeps claims honest |
Takeaway
Correlation is not useless. It is often the first clue. But AI research becomes more credible when causal language is earned rather than assumed.
References and learning path
This note uses the statistics-to-machine-learning route that fits my AI/NLP research goals: build intuition with Seeing Theory and StatQuest, connect it to Python practice with Think Stats, then deepen the ML connection with ISLR/ISLP, CS229, and selected statistical inference references.