· Xiaojing Yang · Statistics · 2 min read
中文Maximum Likelihood Estimation: Why Models Learn Parameters
Maximum likelihood connects probability models to parameter learning.
Core idea
MLE chooses parameters that make the observed data look most plausible under the model.
1. The core question
If a model has parameters, how should we choose them? Maximum likelihood estimation says: choose the parameter values that assign high probability to the data we actually observed.
A probability story with parameters
What we saw
How plausible the data is
Search for best parameters
Use parameters for prediction or analysis
2. From probability to learning
Many ML losses are negative log-likelihoods in disguise. Minimizing cross-entropy for classification can be understood as maximizing the likelihood of the correct labels.
Likelihood view
Choose parameters that explain the observed data.
Loss view
Minimize the penalty for assigning low probability to observed labels.
3. AI/NLP example
Language modeling trains a model to assign high probability to observed token sequences. The next-token prediction loss is not just engineering; it is a likelihood-based learning objective.
4. Caution
MLE depends on the model family and the data. If the data is biased, noisy, or not representative, the fitted parameters inherit that problem.
Takeaway
MLE is a bridge between statistical modeling and ML training: learning means finding parameters that make the observed data probable.
References and learning path
This note uses the statistics-to-machine-learning route that fits my AI/NLP research goals: build intuition with Seeing Theory and StatQuest, connect it to Python practice with Think Stats, then deepen the ML connection with ISLR/ISLP, CS229, and selected statistical inference references.