Note
Click here to download the full example code
1. Lightweight, versatile, and platform agnostic architecture
Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.
Quadratic Function Example
Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).
First of all, import optuna.
import optuna
In optuna, conventionally functions to be optimized are named objective.
def objective(trial):
x = trial.suggest_float("x", -10, 10)
return (x - 2) ** 2
This function returns the value of \((x - 2)^2\). Our goal is to find the value of x
that minimizes the output of the objective function. This is the “optimization.”
During the optimization, Optuna repeatedly calls and evaluates the objective function with
different values of x.
A Trial object corresponds to a single execution of the objective
function and is internally instantiated upon each invocation of the function.
The suggest APIs (for example, suggest_float()) are called
inside the objective function to obtain parameters for a trial.
suggest_float() selects parameters uniformly within the range
provided. In our example, from \(-10\) to \(10\).
To start the optimization, we create a study object and pass the objective function to method
optimize() as follows.
study = optuna.create_study()
study.optimize(objective, n_trials=100)
You can get the best parameter as follows.
best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Found x: 1.9997544157360778, (x - 2)^2: 6.031163068620034e-08
We can see that the x value found by Optuna is close to the optimal value of 2.
Note
When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.
Study Object
Let us clarify the terminology in Optuna as follows:
Trial: A single call of the objective function
Study: An optimization session, which is a set of trials
Parameter: A variable whose value is to be optimized, such as
xin the above example
In Optuna, we use the study object to manage optimization.
Method create_study() returns a study object.
A study object has useful properties for analyzing the optimization outcome.
To get the dictionary of parameter name and parameter values:
study.best_params
{'x': 1.9997544157360778}
To get the best observed value of the objective function:
study.best_value
6.031163068620034e-08
To get the best trial:
study.best_trial
FrozenTrial(number=92, values=[6.031163068620034e-08], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 795382), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 798824), params={'x': 1.9997544157360778}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None)
To get all trials:
study.trials
[FrozenTrial(number=0, values=[125.2764996942231], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 483308), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 483747), params={'x': -9.192698499210238}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[87.32836840494411], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 484206), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 484508), params={'x': -7.344964869112356}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[88.03209402007853], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 484887), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 485173), params={'x': -7.382541980725614}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[47.72259731109812], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 485543), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 485822), params={'x': -4.908154407010466}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[51.67985470730038], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 486220), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 486507), params={'x': 9.188870196859892}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[52.41590877752263], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 486876), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 487163), params={'x': 9.239883201925473}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[12.124612704865578], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 487538), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 487828), params={'x': 5.482041456511622}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[2.108152123566998], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 488203), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 488493), params={'x': 3.451947700010919}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[26.495080960280028], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 488859), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 489154), params={'x': -3.1473372689459573}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[41.256534222834496], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 489525), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 489817), params={'x': 8.423124957747163}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[0.12195517818365643], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 490218), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 493912), params={'x': 1.650779184206244}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[0.36872842420131796], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 494398), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 497520), params={'x': 2.6072301245831913}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[1.9961649345895727], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 497951), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 501119), params={'x': 0.5871429886256809}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[1.812931939942966], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 501545), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 504635), params={'x': 0.6535483893050721}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[3.348981463730014], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 505062), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 508180), params={'x': 3.830022257714374}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[16.717615335093367], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 508607), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 511691), params={'x': -2.0887180552213875}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[0.9230294488516524], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 512124), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 515209), params={'x': 2.9607442161427007}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[20.61454653336603], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 515641), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 518792), params={'x': 6.540324496483267}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[9.477984637383852], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 519215), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 522310), params={'x': -1.0786335665979885}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[0.04004706362394386], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 522740), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 525795), params={'x': 1.7998823755289308}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[0.26517827556430656], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 526245), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 529351), params={'x': 1.485045365527888}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[1.878824984623896], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 529778), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 532918), params={'x': 0.6292976309118393}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[10.724245547788573], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 533352), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 536491), params={'x': 5.274789389836936}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[0.047961378875452564], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 536958), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 540794), params={'x': 1.7809991349892595}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[13.881065213110569], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 541224), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 544371), params={'x': -1.7257301583864828}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[33.564102517600865], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 544799), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 547909), params={'x': -3.7934534189549556}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[6.468566530649236], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 548355), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 551467), params={'x': -0.5433376753095991}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[6.413088330648913], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 551894), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 555086), params={'x': 4.5324076154223105}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[26.039899952278358], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 555514), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 558706), params={'x': 7.102930525911396}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[0.005471465454286689], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 559134), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 562330), params={'x': 1.9260306451678353}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[0.006695131281881252], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 562766), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 565933), params={'x': 2.081823781884494}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.04857718212621632], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 566378), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 569574), params={'x': 2.2204023187859336}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[5.283125803588961], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 570014), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 573208), params={'x': 4.298505123681251}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[131.98925863282048], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 573635), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 576832), params={'x': -9.488657825560846}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[6.638572809214283], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 577267), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 580489), params={'x': -0.5765428017431193}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[76.01313594164668], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 580919), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 584129), params={'x': -6.718551252452823}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[0.2197197490949997], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 584560), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 587767), params={'x': 1.531257267688341}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[17.831882167370537], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 588212), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 591419), params={'x': 6.222781330754712}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[0.6418624059210738], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 591852), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 595100), params={'x': 2.8011631581151706}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[22.611566671825692], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 595555), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 598786), params={'x': -2.755162107838774}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[49.45210712916966], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 599220), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 602475), params={'x': -5.032219217940355}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.0029888813619107233], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 602931), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 606160), params={'x': 1.945329337283048}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[2.2933117605973923], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 606614), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 609889), params={'x': 0.485631563787269}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[0.04524770102398259], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 610372), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 613554), params={'x': 2.212715070044373}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[2.4800991798026537], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 614010), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 617246), params={'x': 3.5748330641063686}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[9.087723262528751], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 617712), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 621004), params={'x': 5.014585089614946}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[4.301309389587277], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 621441), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 624698), params={'x': -0.0739598331663216}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[1.9607204673773964], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 625148), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 628408), params={'x': 3.4002572861361573}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[0.6059974521087018], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 628846), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 632101), params={'x': 1.22154161825522}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[0.15089389370913875], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 632536), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 635773), params={'x': 2.388450632267652}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[12.232221334341903], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 636242), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 639484), params={'x': -1.497459268432143}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.02003270687266356], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 639937), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 643177), params={'x': 2.1415369452569313}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[0.20479637327174136], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 643635), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 647137), params={'x': 2.4525443329351737}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[4.752632337746037], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 647598), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 650882), params={'x': 4.180053287822579}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[4.141113591574467], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 651341), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 654549), params={'x': -0.03497262673837098}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[0.28631012732783057], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 655008), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 658230), params={'x': 1.4649204476642463}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[1.0847344203844098], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 658687), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 661905), params={'x': 3.0415058427029633}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[1.892860395955644], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 662386), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 665624), params={'x': 0.624187368877708}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[16.405151549162778], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 666080), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 669307), params={'x': 6.050327338520034}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[35.47009363096232], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 669771), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 672996), params={'x': 7.9556774283839715}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[0.002279646772341111], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 673468), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 676707), params={'x': 1.9522543533676513}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.06791689656686103], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 677171), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 680669), params={'x': 2.2606087039353464}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[1.618549886036281], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 681146), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 684409), params={'x': 3.2722224200336516}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[0.5922907438162334], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 684883), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 688140), params={'x': 1.2303957225845004}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[0.011078630928785041], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 688621), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 691866), params={'x': 1.8947449244511931}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[7.651415399550495], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 692355), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 695644), params={'x': -0.7661191947474886}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[8.545299232745672], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 696126), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 699415), params={'x': 4.923234378688385}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[3.7302876298504146], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 699881), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 703139), params={'x': 3.931395254692942}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[0.7962690293205721], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 703579), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 707160), params={'x': 1.10766092245124}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[0.013452958927682659], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 707600), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 710909), params={'x': 1.884013108811027}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[3.553206907873867], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 711348), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 714648), params={'x': 0.11500479897855787}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.0064698253926434], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 715097), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 718409), params={'x': 1.9195647751750304}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.012643042954931254], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 718891), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 722521), params={'x': 1.887558713299201}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[1.2954215591502392], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 723177), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 726445), params={'x': 0.8618341249403763}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[0.6035551715027532], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 726890), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 730219), params={'x': 2.776888133197279}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[0.05937768108060019], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 730670), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 734057), params={'x': 1.7563246399805672}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[3.4283725156899223], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 734534), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 737847), params={'x': 0.14841351385091328}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[2.2301188498021336], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 738326), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 741610), params={'x': 3.4933582456336905}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[6.321335196634581], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 742057), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 745372), params={'x': 4.51422656032319}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[0.017762368809936053], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 745817), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 749164), params={'x': 1.8667244628225568}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[0.7605476959102747], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 749611), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 752958), params={'x': 2.8720938572827324}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.002403308469114219], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 753409), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 756746), params={'x': 1.9509764498519924}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[1.1884072483766088], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 757195), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 760549), params={'x': 0.9098590694884405}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[60.93687862798425], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 761003), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 764372), params={'x': 9.806207698235056}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[109.96946641054473], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 764824), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 768181), params={'x': -8.486632748911575}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[0.05635262200135926], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 768632), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 771975), params={'x': 1.7626129278975806}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[3.4369440067582695], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 772438), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 775823), params={'x': 3.8538996754836194}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[6.0660603454159245], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 776282), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 779646), params={'x': -0.46293734094392347}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[0.36974174299551266], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 780129), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 783454), params={'x': 2.6080639300234085}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[2.763743189947601], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 783900), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 787296), params={'x': 0.33754904134058705}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[0.5159875911475527], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 787743), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 791090), params={'x': 1.2816772374847414}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.005674762512575395], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 791539), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 794932), params={'x': 1.9246689804092936}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[6.031163068620034e-08], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 795382), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 798824), params={'x': 1.9997544157360778}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[1.2222553027394607], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 799282), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 802656), params={'x': 3.105556557910748}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[0.011518039445835094], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 803111), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 806488), params={'x': 2.1073221293388977}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[1.2800313000799999], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 806941), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 810321), params={'x': 0.8686153173743247}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[0.17780176049308474], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 810776), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 814214), params={'x': 2.421665460398507}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[9.82187924060899], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 814669), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 818308), params={'x': -1.1339877537426644}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[0.46170645184048204], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 818784), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 822160), params={'x': 1.320510153246951}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[1.869866048088801], datetime_start=datetime.datetime(2022, 12, 19, 8, 18, 45, 822614), datetime_complete=datetime.datetime(2022, 12, 19, 8, 18, 45, 826264), params={'x': 3.367430454571201}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]
To get the number of trials:
len(study.trials)
100
By executing optimize() again, we can continue the optimization.
study.optimize(objective, n_trials=100)
To get the updated number of trials:
len(study.trials)
200
As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:
best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Found x: 1.9997544157360778, (x - 2)^2: 6.031163068620034e-08
Total running time of the script: ( 0 minutes 0.779 seconds)