· Xiaojing Yang · Statistics
Multiple Comparisons in AI Experiments
When we try many models, prompts, seeds, and metrics, false discoveries become easier than they look.
When we try many models, prompts, seeds, and metrics, false discoveries become easier than they look.
Why NLP evaluation needs metrics, uncertainty, human judgment, and task-specific error analysis.
How models learn noise, how validation curves reveal it, and how regularization controls it.
Why LoRA and adapters are useful when full fine-tuning is too expensive or unstable.
PCA explains how variance, projection, and representation are connected.
Why preprocessing belongs inside the validation pipeline, not before the split.