1. Lightweight, versatile, and platform agnostic architecture

Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.

Quadratic Function Example

Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).

First of all, import optuna.

import optuna

In optuna, conventionally functions to be optimized are named objective.

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Found x: 1.9994290848971703, (x - 2)^2: 3.259440546390405e-07

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the dictionary of parameter name and parameter values:

{'x': 1.9994290848971703}

To get the best observed value of the objective function:

study.best_value
3.259440546390405e-07

To get the best trial:

study.best_trial
FrozenTrial(number=89, values=[3.259440546390405e-07], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 785039), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 788438), params={'x': 1.9994290848971703}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None)

To get all trials:

study.trials
[FrozenTrial(number=0, values=[0.13017551150248438], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 488230), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 488647), params={'x': 2.3607984361142442}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[132.514055189316], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 489112), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 489391), params={'x': -9.511474935442287}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[116.80187290349473], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 489795), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 490059), params={'x': -8.807491517623076}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[17.503725384256622], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 490441), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 490690), params={'x': -2.183745377560234}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[47.769401605581784], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 491073), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 491321), params={'x': -4.911541188879784}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[0.047033280524362314], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 491732), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 491988), params={'x': 1.7831284238901688}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[0.20119414954144224], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 492327), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 492574), params={'x': 2.4485467083163606}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[72.82596317570865], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 492953), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 493221), params={'x': -6.533812933015854}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[1.4616769389705775], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 493600), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 493860), params={'x': 3.208998320499486}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[91.86452013853193], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 494232), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 494499), params={'x': -7.584598068700217}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[47.170821749514495], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 494874), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 498412), params={'x': 8.868101757364585}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[15.950056814682197], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 498850), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 501943), params={'x': 5.993752222494805}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[6.469614617063732], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 502453), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 505487), params={'x': -0.5435437124342353}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[0.8804054946044821], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 505916), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 508983), params={'x': 2.938299256423281}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[22.222231476282882], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 509411), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 512575), params={'x': -2.7140461894515715}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[15.037460969755871], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 513048), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 516085), params={'x': 5.877816520898826}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[2.2630857439081558], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 516516), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 519563), params={'x': 0.4956444090880123}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[13.707648787733993], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 520000), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 523035), params={'x': 5.70238420314991}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[39.21532287319443], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 523483), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 526538), params={'x': 8.26221389551606}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[1.41238702962341], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 526975), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 530044), params={'x': 0.8115610955444912}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[30.736044828772027], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 530472), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 533619), params={'x': -3.544009814996004}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[1.7544996742609624], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 534056), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 537170), params={'x': 3.3245752807073528}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[0.01424725756712381], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 537601), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 540691), params={'x': 2.1193618765231337}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[7.048279825175025], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 541131), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 544231), params={'x': -0.6548596620490179}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[0.48256202636784085], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 544660), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 547749), params={'x': 1.3053331544057678}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[6.919307490416777], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 548178), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 551290), params={'x': 4.630457657978318}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[28.002410587308], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 551747), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 554811), params={'x': 7.2917303963172575}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[8.863738172922176], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 555242), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 558349), params={'x': -0.9772030788849753}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[0.058359097962283814], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 558779), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 561943), params={'x': 1.7584237222691685}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[7.062866597751357], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 562375), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 565539), params={'x': 4.657605425519627}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[0.19415339956422403], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 565974), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 569295), params={'x': 1.5593715856141095}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.05271963667362875], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 569728), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 572914), params={'x': 1.770392428971454}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[3.92414805896978], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 573368), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 576527), params={'x': 3.980946253427836}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[13.192953184970769], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 576958), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 580127), params={'x': -1.6322105094516162}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[0.5108328232997884], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 580572), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 583742), params={'x': 1.285274302057224}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[0.011514480971266446], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 584176), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 587340), params={'x': 2.1073055495827986}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[2.4851451368381934], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 587794), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 590958), params={'x': 0.42356568901898317}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[39.58988530854754], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 591389), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 594555), params={'x': -4.292049372704217}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[0.09921101739885295], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 594986), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 598181), params={'x': 2.314977804613044}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[4.31968939644347], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 598614), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 601806), params={'x': 4.078386248136633}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[13.39539306379488], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 602238), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 605492), params={'x': -1.659971729917443}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.06764702136662491], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 605926), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 609134), params={'x': 2.2600904099858834}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[0.0016493125867049101], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 609570), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 612775), params={'x': 2.0406117296689628}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[1.7747913759109364], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 613231), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 616457), params={'x': 3.33221296192123}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[3.61301895247558], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 616892), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 620103), params={'x': 0.09920570484979097}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[0.2592082316963241], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 620548), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 623760), params={'x': 2.5091249666794235}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[9.1249590572221], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 624195), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 627378), params={'x': 5.020754716494224}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[3.354758654925002], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 627843), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 631029), params={'x': 3.8316000259131364}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[5.4721537334702415], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 631487), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 634680), params={'x': -0.3392635023592878}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[26.849383514298456], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 635120), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 638336), params={'x': 7.181639076035541}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[21.411539075545324], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 638775), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 642000), params={'x': -2.6272604287575305}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.11594810818136277], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 642439), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 645904), params={'x': 2.3405115389841624}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[0.19697336990685846], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 646348), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 649593), params={'x': 1.5561831797837553}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[1.5781141038545767], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 650032), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 653261), params={'x': 0.7437698841953451}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[0.0734229115185451], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 653699), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 656914), params={'x': 1.7290333756372473}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[139.5899786639221], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 657369), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 660602), params={'x': -9.814820297572117}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[1.0172476193654052], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 661055), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 664268), params={'x': 3.0085869418971303}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[4.121931064687284], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 664713), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 667929), params={'x': -0.03025394093627698}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[8.923971130261144], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 668373), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 671603), params={'x': -0.9873016470154372}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[1.1193027971453124], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 672047), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 675251), params={'x': 0.9420289242397444}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[9.657876232592503], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 675713), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 679209), params={'x': 5.107712379322209}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.00946460708111237], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 679677), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 682886), params={'x': 2.097286212184011}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[0.06158109992870894], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 683331), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 686576), params={'x': 1.751844605279859}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[1.003679654700249], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 687018), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 690255), params={'x': 3.001838137974518}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[3.2596723023809324], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 690698), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 693953), params={'x': 3.8054562587836163}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[80.01032155812482], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 694398), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 697724), params={'x': -6.9448488840295575}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[0.0026271802685901777], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 698168), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 701449), params={'x': 2.0512560266562887}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[0.3589879506526501], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 701894), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 705171), params={'x': 2.59915603197552}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[1.4595781392899025], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 705627), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 708862), params={'x': 0.7918699824563986}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[3.176153980086128], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 709322), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 712559), params={'x': 0.2178232466766581}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[12.846025439487361], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 713014), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 716294), params={'x': 5.584135242912488}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[1.8101072811072927], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 716742), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 720018), params={'x': 3.345402274826118}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.6714934802412658], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 720465), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 723751), params={'x': 1.1805529423804941}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[0.013915416262409051], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 724198), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 727482), params={'x': 1.882036377376714}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[0.002401641760078424], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 727931), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 731197), params={'x': 2.0490065481346975}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[0.029727124908989183], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 731665), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 734933), params={'x': 2.1724155587787517}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[0.049361028757063295], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 735377), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 738666), params={'x': 2.2221734204558756}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[0.5732153752178301], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 739113), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 742403), params={'x': 2.7571098831859415}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[5.6597769287033355], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 742848), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 746152), params={'x': 4.379028568282301}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[2.9841375228878873], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 746600), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 749911), params={'x': 3.7274656358052067}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[7.479343236770323], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 750361), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 753944), params={'x': -0.7348387953900177}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.5558795832499771], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 754392), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 758006), params={'x': 1.2544266747998711}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[0.0021734685081564852], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 758455), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 761779), params={'x': 2.0466204730580513}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[0.000712171229575699], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 762228), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 765563), params={'x': 2.026686536485196}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[1.95323642695698], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 766011), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 769359), params={'x': 3.397582350688853}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[0.0048522367860629285], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 769806), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 773145), params={'x': 2.069657998722781}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[2.1060008569164848], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 773592), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 776917), params={'x': 0.548793310063489}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[0.7682602328928871], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 777382), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 780761), params={'x': 2.8765045538346548}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[5.211715884639394], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 781228), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 784578), params={'x': -0.28291828251459594}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[3.259440546390405e-07], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 785039), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 788438), params={'x': 1.9994290848971703}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[0.4623399172817482], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 788896), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 792247), params={'x': 1.3200441799044969}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.006583790069247623], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 792703), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 796346), params={'x': 2.08114055748667}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[0.29771185346137136], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 796805), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 800162), params={'x': 2.5456297769196357}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[0.0003391846015870378], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 800618), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 803961), params={'x': 2.0184169650482113}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[0.8638638508534322], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 804418), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 807779), params={'x': 1.0705572363757776}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[118.68107629088598], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 808234), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 811904), params={'x': -8.894084463179363}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[2.0936164544753786], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 812494), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 815840), params={'x': 3.446933465808079}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[5.204095227073554], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 816295), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 819661), params={'x': 4.281248611412959}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[2.4675502484971807], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 820115), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 823485), params={'x': 0.42915619856804965}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[0.02246070731266203], datetime_start=datetime.datetime(2022, 9, 15, 6, 58, 41, 823944), datetime_complete=datetime.datetime(2022, 9, 15, 6, 58, 41, 827292), params={'x': 1.8501310328564915}, distributions={'x': FloatDistribution(high=10.0, log=False, low=-10.0, step=None)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]

To get the number of trials:

len(study.trials)
100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)
200

As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
Found x: 1.9994290848971703, (x - 2)^2: 3.259440546390405e-07

Total running time of the script: ( 0 minutes 0.772 seconds)

Gallery generated by Sphinx-Gallery