· Xiaojing Yang · Machine Learning · 2 min read
中文Grid Search and Randomized Search
How to tune hyperparameters without confusing search effort with scientific evidence.
Core idea
Hyperparameter search is useful, but every extra trial is another chance to overfit validation data.
1. Parameters vs hyperparameters
Parameters are learned from data. Hyperparameters are chosen outside training: regularization strength, tree depth, learning rate, number of neighbors, batch size, or LoRA rank.
What values are allowed?
Grid or random
Score each setting
Pick best validation setting
Use untouched test data
2. Grid vs random
| Method | Strength | Weakness |
|---|---|---|
| Grid search | systematic over small spaces | expensive, wastes trials |
| Randomized search | efficient in large spaces | less exhaustive |
| Successive halving | allocates resources adaptively | more moving parts |
3. sklearn example
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
param_grid = {"C": [0.01, 0.1, 1, 10], "penalty": ["l2"]}
search = GridSearchCV(model, param_grid, cv=5, scoring="f1_macro")
search.fit(X_train, y_train)4. AI/NLP connection
In NLP fine-tuning, hyperparameters include learning rate, batch size, epochs, warmup, dropout, rank, alpha, and decoding settings. A clean search log is part of research credibility.
Good report
State search space, budget, metric, validation protocol, and final test result.
Bad report
Only report the best number after many hidden trials.
Takeaway
Hyperparameter tuning is not a magic path to better models. It is controlled search under a fair evaluation protocol.
Interview pattern
When this appears in an interview, I would answer in four layers:
- give the short definition;
- explain the intuition;
- name the common failure mode;
- connect it to a real evaluation or deployment decision.