Note
Go to the end to download the full example code
Saving/Resuming Study with RDB Backend
An RDB backend enables persistent experiments (i.e., to save and resume a study) as well as access to history of studies. In addition, we can run multi-node optimization tasks with this feature, which is described in Easy Parallelization.
In this section, let’s try simple examples running on a local environment with SQLite DB.
Note
You can also utilize other RDB backends, e.g., PostgreSQL or MySQL, by setting the storage argument to the DB’s URL. Please refer to SQLAlchemy’s document for how to set up the URL.
New Study
We can create a persistent study by calling create_study()
function as follows.
An SQLite file example.db
is automatically initialized with a new study record.
import logging
import sys
import optuna
# Add stream handler of stdout to show the messages
optuna.logging.get_logger("optuna").addHandler(logging.StreamHandler(sys.stdout))
study_name = "example-study" # Unique identifier of the study.
storage_name = "sqlite:///{}.db".format(study_name)
study = optuna.create_study(study_name=study_name, storage=storage_name)
A new study created in RDB with name: example-study
To run a study, call optimize()
method passing an objective function.
def objective(trial):
x = trial.suggest_float("x", -10, 10)
return (x - 2) ** 2
study.optimize(objective, n_trials=3)
Trial 0 finished with value: 133.28431999630033 and parameters: {'x': -9.544882848963878}. Best is trial 0 with value: 133.28431999630033.
Trial 1 finished with value: 46.70039844798285 and parameters: {'x': -4.83376897824201}. Best is trial 1 with value: 46.70039844798285.
Trial 2 finished with value: 8.245035138547546 and parameters: {'x': 4.871416921756147}. Best is trial 2 with value: 8.245035138547546.
Resume Study
To resume a study, instantiate a Study
object
passing the study name example-study
and the DB URL sqlite:///example-study.db
.
study = optuna.create_study(study_name=study_name, storage=storage_name, load_if_exists=True)
study.optimize(objective, n_trials=3)
Using an existing study with name 'example-study' instead of creating a new one.
Trial 3 finished with value: 28.038706648384558 and parameters: {'x': -3.295158793500395}. Best is trial 2 with value: 8.245035138547546.
Trial 4 finished with value: 125.72890071810274 and parameters: {'x': -9.212889936055857}. Best is trial 2 with value: 8.245035138547546.
Trial 5 finished with value: 50.530922261499974 and parameters: {'x': -5.108510551550161}. Best is trial 2 with value: 8.245035138547546.
Note that the storage doesn’t store the state of the instance of samplers
and pruners
.
When we resume a study with a sampler whose seed
argument is specified for
reproducibility, you need to restore the sampler with using pickle
as follows:
import pickle
# Save the sampler with pickle to be loaded later.
with open("sampler.pkl", "wb") as fout:
pickle.dump(study.sampler, fout)
restored_sampler = pickle.load(open("sampler.pkl", "rb"))
study = optuna.create_study(
study_name=study_name, storage=storage_name, load_if_exists=True, sampler=restored_sampler
)
study.optimize(objective, n_trials=3)
Experimental History
We can access histories of studies and trials via the Study
class.
For example, we can get all trials of example-study
as:
study = optuna.create_study(study_name=study_name, storage=storage_name, load_if_exists=True)
df = study.trials_dataframe(attrs=("number", "value", "params", "state"))
Using an existing study with name 'example-study' instead of creating a new one.
The method trials_dataframe()
returns a pandas dataframe like:
print(df)
number value params_x state
0 0 133.284320 -9.544883 COMPLETE
1 1 46.700398 -4.833769 COMPLETE
2 2 8.245035 4.871417 COMPLETE
3 3 28.038707 -3.295159 COMPLETE
4 4 125.728901 -9.212890 COMPLETE
5 5 50.530922 -5.108511 COMPLETE
A Study
object also provides properties
such as trials
, best_value
,
best_params
(see also Lightweight, versatile, and platform agnostic architecture).
print("Best params: ", study.best_params)
print("Best value: ", study.best_value)
print("Best Trial: ", study.best_trial)
print("Trials: ", study.trials)
Best params: {'x': 4.871416921756147}
Best value: 8.245035138547546
Best Trial: FrozenTrial(number=2, state=1, values=[8.245035138547546], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 98639), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 126670), params={'x': 4.871416921756147}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=3, value=None)
Trials: [FrozenTrial(number=0, state=1, values=[133.28431999630033], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 43, 961745), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 13322), params={'x': -9.544882848963878}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=1, value=None), FrozenTrial(number=1, state=1, values=[46.70039844798285], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 46181), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 74401), params={'x': -4.83376897824201}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=2, value=None), FrozenTrial(number=2, state=1, values=[8.245035138547546], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 98639), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 126670), params={'x': 4.871416921756147}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=3, value=None), FrozenTrial(number=3, state=1, values=[28.038706648384558], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 204751), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 245702), params={'x': -3.295158793500395}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=4, value=None), FrozenTrial(number=4, state=1, values=[125.72890071810274], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 272732), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 300667), params={'x': -9.212889936055857}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=5, value=None), FrozenTrial(number=5, state=1, values=[50.530922261499974], datetime_start=datetime.datetime(2024, 3, 18, 7, 19, 44, 323936), datetime_complete=datetime.datetime(2024, 3, 18, 7, 19, 44, 352165), params={'x': -5.108510551550161}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, trial_id=6, value=None)]
Total running time of the script: (0 minutes 1.647 seconds)