When instantiating an estimator or an optimizer, default values for any hyperparameters that are not specified will be used. Relying on the default
values can lead to non-reproducible results across different versions of the library.
Furthermore, the default values might not be the best choice for the specific problem at hand and can lead to suboptimal performance.
Here are the estimators and the parameters considered by this rule :
Scikit-learn - Estimator |
Hyperparameters |
AdaBoostClassifier |
learning_rate |
AdaBoostRegressor |
learning_rate |
GradientBoostingClassifier |
learning_rate |
GradientBoostingRegressor |
learning_rate |
HistGradientBoostingClassifier |
learning_rate |
HistGradientBoostingRegressor |
learning_rate |
RandomForestClassifier |
min_samples_leaf, max_features |
RandomForestRegressor |
min_samples_leaf, max_features |
ElasticNet |
alpha, l1_ratio |
NearestNeighbors |
n_neighbors |
KNeighborsClassifier |
n_neighbors |
KNeighborsRegressor |
n_neighbors |
NuSVC |
nu, kernel, gamma |
NuSVR |
C, kernel, gamma |
SVC |
C, kernel, gamma |
SVR |
C, kernel, gamma |
DecisionTreeClassifier |
ccp_alpha |
DecisionTreeRegressor |
ccp_alpha |
MLPClassifier |
hidden_layer_sizes |
MLPRegressor |
hidden_layer_sizes |
PolynomialFeatures |
degree, interaction_only |
PyTorch - Optimizer |
Hyperparameters |
Adadelta |
lr, weight_decay |
Adagrad |
lr, weight_decay |
Adam |
lr, weight_decay |
AdamW |
lr, weight_decay |
SparseAdam |
lr |
Adamax |
lr, weight_decay |
ASGD |
lr, weight_decay |
LBFGS |
lr |
NAdam |
lr, weight_decay, momentum_decay |
RAdam |
lr, weight_decay |
RMSprop |
lr, weight_decay, momentum |
Rprop |
lr |
SGD |
lr, weight_decay, momentum |