· Xiaojing Yang · Explainability and Responsible AI · 4 min read
中文From Association to Causal Evidence: What Can Attribution Claim?
Training-data attribution can provide evidence of influence, but causal claims require careful interventions, retraining, controls, and uncertainty analysis.
Core idea
Attribution is not automatically causality. A high attribution score suggests influence under a defined procedure; a causal claim needs a plausible intervention and comparison.
This note is part of my series Training Data Attribution for NLP and LLM Research. The series is written as both a research notebook and an interview preparation path: each article should help me explain the idea clearly, connect it to my thesis, and identify what would become future PhD work.
Guiding question: When can an attribution result be interpreted causally?
Intuition
Similarity can explain why a training example is retrieved. Association can show that a data group correlates with a behaviour. Attribution estimates contribution under a chosen method. Causal evidence asks whether changing the data would change the model behaviour in a predictable way.
In NLP and LLM research, this matters because model behaviour is deeply shaped by data mixture. A model may be fluent because of broad web text, domain-accurate because of specialised documents, safer because of curated instruction data, or biased because of repeated patterns in a subset of the corpus. Training-data attribution gives us language for asking these questions systematically instead of only saying “the data matters”.
Formal lens
A causal-style statement requires an intervention such as deleting, correcting, reweighting, or adding counterfactual examples, plus a comparison model trained or evaluated under controlled conditions. The estimand should be tied to a concrete intervention, not merely to a ranking score.
The important discipline is to define the attribution setup before interpreting the score:
| Design choice | Question to answer |
|---|---|
| Attribution unit | What receives credit: source, group, document, example, or token? |
| Utility function | Which behaviour is being explained: quality, terminology, style, factuality, or safety? |
| Intervention | Are we adding, deleting, reweighting, correcting, or retraining? |
| Estimator | Is the score exact, sampled, gradient-based, surrogate-based, or heuristic? |
| Uncertainty | How stable is the score across seeds, samples, metrics, and evaluation sets? |
NLP / LLM example
If a high-attribution data group is removed and terminology performance drops more than under random group removal, that supports a causal influence claim. If a suspicious example is only semantically similar to a bad output, it is evidence for diagnosis, not proof that the example caused the error.
This is why I do not want to treat attribution as a generic interpretability topic. For my profile, the natural connection is multilingual and domain-specific NLP: low-resource settings, technical terminology, written-standard variation, and evaluation beyond one headline metric.
Connection to my thesis
In my thesis narrative, training-data attribution is useful because it turns a vague data question into an experimental design:
- define interpretable data units;
- define the model behaviour to explain;
- compare controlled data coalitions or interventions;
- estimate contribution;
- report uncertainty and limitations;
- decide what evidence is strong enough to support a causal-style claim.
That structure helps me avoid overclaiming. A score is not automatically a causal explanation. It is a measurement produced by a specific setup.
What I have done, understand, and would extend
| Level | Status |
|---|---|
| Already completed / thesis-ready | Group-level attribution, coalition thinking, metric-based utilities, cautious interpretation, random baselines, bootstrap-style reliability checks. |
| I understand but may not fully implement yet | Instance-level gradient attribution, influence functions, TracIn, Monte Carlo Shapley, surrogate/datamodel approximations. |
| Strong PhD extension | Hierarchical attribution, intervention-based validation, factuality/style-specific utilities, scalable attribution for LLM data mixtures. |
Interview answer
I would be careful with language. I can say a data group has high estimated contribution under a Shapley-style coalition analysis. I would only make stronger causal claims if I validate them through deletion, correction, reweighting, or counterfactual retraining experiments.
References and reading path
- Lloyd Shapley, A Value for n-Person Games.
- Ghorbani and Zou, Data Shapley: Equitable Valuation of Data for Machine Learning.
- Koh and Liang, Understanding Black-box Predictions via Influence Functions.
- Pruthi et al., Estimating Training Data Influence by Tracing Gradient Descent.
- Ilyas et al., Datamodels: Predicting Predictions from Training Data.
- Rei et al., COMET: A Neural Framework for MT Evaluation.