Re-use the best values¶
In some cases, you may want to re-evaluate the objective function with the best hyperparameters again after the hyperparameter optimization.
You have found good hyperparameters with Optuna and want to run a similar objective function using the best hyperparameters found so far to further analyze the results, or
You have optimized with Optuna using a partial dataset to reduce training time. After the hyperparameter tuning, you want to train the model using the whole dataset with the best hyperparameter values found.
best_trial provides an interface to re-evaluate the objective function with the current best hyperparameter values.
This tutorial shows an example of how to re-run a different objective function with the current best values, like the first example above.
Investigating the best model further¶
Let’s consider a classical supervised classification problem with Optuna as follows:
from sklearn import metrics from sklearn.datasets import make_classification from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split import optuna def objective(trial): X, y = make_classification(n_features=10, random_state=1) X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1) C = trial.suggest_loguniform("C", 1e-7, 10.0) clf = LogisticRegression(C=C) clf.fit(X_train, y_train) return clf.score(X_test, y_test) study = optuna.create_study(direction="maximize") study.optimize(objective, n_trials=10) print(study.best_trial.value) # Show the best value.
Suppose after the hyperparameter optimization, you want to calculate other evaluation metrics
such as recall, precision, and f1-score on the same dataset.
You can define another objective function that shares most of the
function to reproduce the model with the best hyperparameters.
def detailed_objective(trial): # Use same code objective to reproduce the best model X, y = make_classification(n_features=10, random_state=1) X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1) C = trial.suggest_loguniform("C", 1e-7, 10.0) clf = LogisticRegression(C=C) clf.fit(X_train, y_train) # calculate more evaluation metrics pred = clf.predict(X_test) acc = metrics.accuracy_score(pred, y_test) recall = metrics.recall_score(pred, y_test) precision = metrics.precision_score(pred, y_test) f1 = metrics.f1_score(pred, y_test) return acc, f1, recall, precision
study.best_trial as the argument of
detailed_objective(study.best_trial) # calculate acc, f1, recall, and precision
(0.92, 0.9285714285714286, 0.9285714285714286, 0.9285714285714286)
The difference between
best_trial and ordinal trials¶
best_trial, which returns the best_trial as a
FrozenTrial is different from an active trial
and behaves differently from
Trial in some situations.
For example, pruning does not work because
Total running time of the script: ( 0 minutes 0.057 seconds)