First Optimization

Quadratic Function Example

Usually, Optuna is used to optimize hyper-parameters, but as an example, let us directly optimize a quadratic function in an IPython shell.

import optuna

The objective function is what will be optimized.

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

print(study.best_params)

Out:

{'x': 2.0252821005120367}

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyper-parameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the best parameter:

study.best_params

Out:

{'x': 2.0252821005120367}

To get the best value:

study.best_value

Out:

0.0006391846063007274

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=21, value=0.0006391846063007274, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 959119), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 962382), params={'x': 2.0252821005120367}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, value=138.72896497157504, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 906293), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 906526), params={'x': -9.778326068316119}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE), FrozenTrial(number=1, value=142.6636328525799, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 907369), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 907505), params={'x': -9.944188245861662}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE), FrozenTrial(number=2, value=4.545491956100232, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 908234), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 908351), params={'x': -0.13201593711215764}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE), FrozenTrial(number=3, value=4.803566591396435, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 909072), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 909189), params={'x': -0.19170403827625293}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE), FrozenTrial(number=4, value=17.655154324649434, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 909904), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 910021), params={'x': 6.201803698966604}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE), FrozenTrial(number=5, value=113.24265153988458, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 910734), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 910853), params={'x': -8.641553060521034}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE), FrozenTrial(number=6, value=48.521998409565974, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 911551), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 911669), params={'x': 8.965773353301554}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE), FrozenTrial(number=7, value=1.2169672425351452, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 912378), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 912498), params={'x': 0.8968376173313626}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE), FrozenTrial(number=8, value=9.492862929625238, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 913188), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 913311), params={'x': -1.0810489982512834}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE), FrozenTrial(number=9, value=16.127719969549396, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 914004), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 914127), params={'x': 6.015933262586593}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE), FrozenTrial(number=10, value=37.214845520326904, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 914810), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 918500), params={'x': -4.100397160868045}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE), FrozenTrial(number=11, value=4.1702618011440515, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 919252), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 922544), params={'x': 4.042121886946039}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE), FrozenTrial(number=12, value=3.235299177263862, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 923289), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 926618), params={'x': 3.798693741931589}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE), FrozenTrial(number=13, value=1.2069424505932707, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 927348), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 930554), params={'x': 3.0986093257356186}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE), FrozenTrial(number=14, value=41.490314356610334, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 931344), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 934575), params={'x': -4.441297567773929}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE), FrozenTrial(number=15, value=0.38637642068077954, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 935312), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 938533), params={'x': 2.6215918441234405}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE), FrozenTrial(number=16, value=58.09454389991968, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 939269), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 942518), params={'x': 9.621977689544865}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE), FrozenTrial(number=17, value=0.3523626797199026, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 943245), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 946466), params={'x': 2.5936014485493635}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE), FrozenTrial(number=18, value=20.63770858326266, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 947199), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 950427), params={'x': -2.542874484647651}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE), FrozenTrial(number=19, value=0.05014866912453467, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 951166), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 954394), params={'x': 1.7760610147282643}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE), FrozenTrial(number=20, value=20.80276822972111, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 955134), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 958389), params={'x': 6.561005177559121}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE), FrozenTrial(number=21, value=0.0006391846063007274, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 959119), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 962382), params={'x': 2.0252821005120367}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE), FrozenTrial(number=22, value=0.4478174661634386, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 963128), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 966392), params={'x': 1.3308083487046192}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE), FrozenTrial(number=23, value=0.18079569981387844, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 967130), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 970362), params={'x': 1.5747992241142093}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE), FrozenTrial(number=24, value=17.452950237531734, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 971103), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 974342), params={'x': -2.177672825573077}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE), FrozenTrial(number=25, value=11.456121567758608, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 975084), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 978331), params={'x': 5.3846892867379434}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE), FrozenTrial(number=26, value=0.11771172761528408, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 979125), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 982396), params={'x': 1.6569085725126842}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE), FrozenTrial(number=27, value=34.17911376798544, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 983225), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 986489), params={'x': 7.846290599002537}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE), FrozenTrial(number=28, value=6.702664015208601, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 987282), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 990639), params={'x': 4.5889503693984945}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE), FrozenTrial(number=29, value=75.46483850549481, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 991431), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 994777), params={'x': -6.687050046217924}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE), FrozenTrial(number=30, value=12.408345104206477, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 995570), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 23, 998909), params={'x': -1.5225480982105093}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE), FrozenTrial(number=31, value=0.03975096876193018, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 23, 999741), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 3105), params={'x': 1.8006235501320926}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE), FrozenTrial(number=32, value=0.17700904249089627, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 3864), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 7234), params={'x': 1.5792755741689148}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE), FrozenTrial(number=33, value=2.2618412160516987, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 8031), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 11414), params={'x': 0.4960581074882917}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE), FrozenTrial(number=34, value=6.114307900558464, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 12162), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 15531), params={'x': -0.47271266033044457}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE), FrozenTrial(number=35, value=0.03621655627869852, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 16341), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 19704), params={'x': 2.1903064798652387}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE), FrozenTrial(number=36, value=1.2962325586875942, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 20442), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 23838), params={'x': 3.138522094070903}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE), FrozenTrial(number=37, value=9.715114842941126, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 24576), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 28005), params={'x': 5.1169078977315205}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE), FrozenTrial(number=38, value=30.04026531821022, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 28794), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 32178), params={'x': 7.480900046361931}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE), FrozenTrial(number=39, value=3.0256197786854373, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 32925), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 36353), params={'x': 0.26056912218811146}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE), FrozenTrial(number=40, value=26.7176295706019, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 37112), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 40531), params={'x': -3.168909901575177}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE), FrozenTrial(number=41, value=0.02113550906085753, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 41323), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 44712), params={'x': 2.1453805663108296}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE), FrozenTrial(number=42, value=0.012758388680731845, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 45454), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 48908), params={'x': 2.1129530375011307}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE), FrozenTrial(number=43, value=8.510178677690368, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 49708), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 53112), params={'x': -0.9172210539639205}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE), FrozenTrial(number=44, value=3.3281424752871946, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 53860), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 57337), params={'x': 3.8243197294573106}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE), FrozenTrial(number=45, value=0.4401895054867577, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 58130), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 61541), params={'x': 2.6634677878290383}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE), FrozenTrial(number=46, value=2.3477824906597444, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 62336), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 65821), params={'x': 0.4677524708260272}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE), FrozenTrial(number=47, value=0.07236255097233141, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 66622), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 70024), params={'x': 2.269002882832752}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE), FrozenTrial(number=48, value=2.521005990584306, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 70769), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 74209), params={'x': 3.5877676122733786}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE), FrozenTrial(number=49, value=5.452141385793526, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 74955), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 78347), params={'x': 4.3349820953903535}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE), FrozenTrial(number=50, value=1.41830341850961, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 79093), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 82530), params={'x': 0.8090745537567812}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE), FrozenTrial(number=51, value=0.0033818131374045773, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 83286), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 87253), params={'x': 2.058153358780079}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE), FrozenTrial(number=52, value=4.094979834863109, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 87995), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 91443), params={'x': -0.023605652014025313}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE), FrozenTrial(number=53, value=0.04987488088933279, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 92192), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 95604), params={'x': 2.223326847668015}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE), FrozenTrial(number=54, value=0.8430977729053409, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 96345), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 99763), params={'x': 2.9182035574453744}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE), FrozenTrial(number=55, value=0.7271720159196858, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 100505), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 103960), params={'x': 1.1472561838865756}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE), FrozenTrial(number=56, value=9.226609793217403, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 104707), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 108122), params={'x': 5.037533504871576}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE), FrozenTrial(number=57, value=2.1964057351784247, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 108865), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 113154), params={'x': 3.4820275757145764}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE), FrozenTrial(number=58, value=6.443682437103768, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 113904), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 121572), params={'x': -0.5384409461525332}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE), FrozenTrial(number=59, value=12.801073189779034, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 122324), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 129920), params={'x': -1.5778587436872118}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE), FrozenTrial(number=60, value=0.0011240196824456098, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 130768), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 138570), params={'x': 2.033526402766262}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE), FrozenTrial(number=61, value=0.024834458381163705, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 139436), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 144170), params={'x': 2.1575895249728347}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE), FrozenTrial(number=62, value=0.028113469109424062, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 144934), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 148388), params={'x': 2.1676707163145195}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE), FrozenTrial(number=63, value=1.1489093259822603, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 149140), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 152603), params={'x': 0.9281281205375989}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE), FrozenTrial(number=64, value=1.29604160832546, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 153351), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 156802), params={'x': 3.1384382321081192}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE), FrozenTrial(number=65, value=4.548091885173914, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 157551), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 161016), params={'x': 4.132625584854011}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE), FrozenTrial(number=66, value=17.132331981377284, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 161766), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 165218), params={'x': 6.13912212689808}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE), FrozenTrial(number=67, value=0.005482968199839827, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 165967), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 169430), params={'x': 1.9259529325372042}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE), FrozenTrial(number=68, value=0.6268633641951014, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 170181), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 173655), params={'x': 1.2082529670437019}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE), FrozenTrial(number=69, value=6.475316145281854, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 174403), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 177860), params={'x': 4.54466425001057}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE), FrozenTrial(number=70, value=3.5578990551595973, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 178608), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 182052), params={'x': 0.1137606050239548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE), FrozenTrial(number=71, value=0.000704603670535591, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 182804), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 186258), params={'x': 2.026544371729909}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE), FrozenTrial(number=72, value=0.03678262733800489, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 187009), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 190468), params={'x': 1.8082120250432658}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE), FrozenTrial(number=73, value=0.43826683808284206, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 191240), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 194685), params={'x': 2.6620172490825613}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE), FrozenTrial(number=74, value=0.37672743589001284, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 195458), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 198894), params={'x': 1.3862187393785854}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE), FrozenTrial(number=75, value=1.7342411914056333, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 199656), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 203119), params={'x': 0.6830940840722017}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE), FrozenTrial(number=76, value=2.1857794915748436, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 203900), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 207349), params={'x': 3.478438193356369}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE), FrozenTrial(number=77, value=0.1122032410975363, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 208112), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 211575), params={'x': 2.3349675224518585}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE), FrozenTrial(number=78, value=1.0551719080996047, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 212343), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 215800), params={'x': 3.0272156093535596}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE), FrozenTrial(number=79, value=0.024210659170313488, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 216564), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 220041), params={'x': 1.8444022520397114}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE), FrozenTrial(number=80, value=6.962138295413229, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 220818), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 224325), params={'x': -0.6385864199251139}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE), FrozenTrial(number=81, value=0.015617027474600408, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 225090), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 228588), params={'x': 1.8750318941705508}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE), FrozenTrial(number=82, value=0.06533100691839193, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 229363), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 232856), params={'x': 1.7444006906926548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE), FrozenTrial(number=83, value=1.3481295661936543, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 233632), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 237106), params={'x': 0.838910181685476}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE), FrozenTrial(number=84, value=3.045666532190432, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 237881), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 241377), params={'x': 3.7451838104309907}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE), FrozenTrial(number=85, value=0.43824408259455, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 242141), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 245635), params={'x': 1.3379999376174123}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE), FrozenTrial(number=86, value=4.250661132582886, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 246399), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 250123), params={'x': -0.06171315477757133}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE), FrozenTrial(number=87, value=0.4516789157384228, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 250905), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 254393), params={'x': 2.6720706181186786}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE), FrozenTrial(number=88, value=0.01264737393113938, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 255164), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 258634), params={'x': 1.8875394561139802}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE), FrozenTrial(number=89, value=1.6843105439961488, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 259415), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 262907), params={'x': 3.2978099028733556}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE), FrozenTrial(number=90, value=2.657055739901121, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 263654), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 267126), params={'x': 0.3699522277242544}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE), FrozenTrial(number=91, value=0.007316643772529506, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 267918), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 271421), params={'x': 1.9144626176895183}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE), FrozenTrial(number=92, value=0.5045048988445351, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 272154), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 275662), params={'x': 2.710285082797418}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE), FrozenTrial(number=93, value=0.0007281750747390124, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 276395), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 279915), params={'x': 1.9730152807178023}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE), FrozenTrial(number=94, value=1.1066370866217041, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 280649), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 284183), params={'x': 0.9480318034171831}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE), FrozenTrial(number=95, value=0.09640266819970655, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 284915), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 288423), params={'x': 1.6895122092582278}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE), FrozenTrial(number=96, value=3.6467496618024104, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 289158), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 292788), params={'x': 3.909646475608093}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE), FrozenTrial(number=97, value=10.071377372633606, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 293525), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 297061), params={'x': -1.1735433465817995}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE), FrozenTrial(number=98, value=0.5561671889210295, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 297796), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 301339), params={'x': 1.25423382423106}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE), FrozenTrial(number=99, value=0.18171437518554368, datetime_start=datetime.datetime(2020, 9, 7, 4, 31, 24, 302080), datetime_complete=datetime.datetime(2020, 9, 7, 4, 31, 24, 305634), params={'x': 2.426279691265657}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

Total running time of the script: ( 0 minutes 0.880 seconds)

Gallery generated by Sphinx-Gallery