First Optimization

Quadratic Function Example

Usually, Optuna is used to optimize hyper-parameters, but as an example, let us directly optimize a quadratic function in an IPython shell.

import optuna

The objective function is what will be optimized.

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

print(study.best_params)

Out:

{'x': 1.9768548120705323}

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyper-parameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the best parameter:

study.best_params

Out:

{'x': 1.9768548120705323}

To get the best value:

study.best_value

Out:

0.000535699724290379

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=12, value=0.000535699724290379, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 739335), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 743264), params={'x': 1.9768548120705323}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, value=26.108223413568215, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 726927), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 727134), params={'x': 7.109620672179904}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE), FrozenTrial(number=1, value=18.72372699415082, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 727496), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 727716), params={'x': 6.327092210035605}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE), FrozenTrial(number=2, value=2.787588754776623, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 728019), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 728222), params={'x': 0.33039263454648626}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE), FrozenTrial(number=3, value=70.97380957550165, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 728510), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 728698), params={'x': -6.424595514058917}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE), FrozenTrial(number=4, value=44.40947655102064, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 728958), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 729175), params={'x': -4.664043558607689}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE), FrozenTrial(number=5, value=32.74545216290248, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 729460), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 729666), params={'x': -3.7223642109623256}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE), FrozenTrial(number=6, value=26.651678343423676, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 729941), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 730148), params={'x': -3.1625263528067027}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE), FrozenTrial(number=7, value=51.47233368554702, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 730422), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 730630), params={'x': 9.174422184785826}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE), FrozenTrial(number=8, value=63.408745753113045, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 730901), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 731114), params={'x': -5.962960865979001}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE), FrozenTrial(number=9, value=24.492444658839297, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 731385), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 731590), params={'x': 6.948984204747404}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE), FrozenTrial(number=10, value=0.0014216542654378207, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 731860), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 735369), params={'x': 2.0377048307970984}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE), FrozenTrial(number=11, value=0.04358761164792005, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 735672), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 739016), params={'x': 1.791223536652428}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE), FrozenTrial(number=12, value=0.000535699724290379, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 739335), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 743264), params={'x': 1.9768548120705323}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE), FrozenTrial(number=13, value=0.7550323185361455, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 743561), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 746866), params={'x': 2.8689259568778835}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE), FrozenTrial(number=14, value=3.401616272854961, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 747244), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 750560), params={'x': 3.8443471128979385}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE), FrozenTrial(number=15, value=143.3250814593747, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 751010), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 754191), params={'x': -9.971845365664171}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE), FrozenTrial(number=16, value=11.607849862566022, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 754476), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 757728), params={'x': -1.4070294777952865}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE), FrozenTrial(number=17, value=8.712735356877305, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 758032), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 761373), params={'x': -0.9517342964564586}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE), FrozenTrial(number=18, value=6.728665939633097, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 761670), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 765020), params={'x': 4.59396722023103}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE), FrozenTrial(number=19, value=0.044737055018407884, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 765329), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 768615), params={'x': 1.7884886409234533}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE), FrozenTrial(number=20, value=2.131548662113765, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 768916), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 772103), params={'x': 0.5400175815737489}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE), FrozenTrial(number=21, value=0.002579815917549637, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 772352), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 775609), params={'x': 1.9492081116953737}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE), FrozenTrial(number=22, value=5.691093993835442, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 775919), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 779286), params={'x': 4.385601390390994}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE), FrozenTrial(number=23, value=0.0008400118596813619, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 779584), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 782892), params={'x': 2.028982958090598}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE), FrozenTrial(number=24, value=17.022530632100256, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 783224), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 787599), params={'x': -2.1258369614055592}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE), FrozenTrial(number=25, value=13.281698719724579, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 787901), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 791208), params={'x': 5.644406497596636}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE), FrozenTrial(number=26, value=1.421688270420719, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 791487), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 794764), params={'x': 0.8076542991142548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE), FrozenTrial(number=27, value=44.7859280722061, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 795088), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 798426), params={'x': 8.692228931544863}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE), FrozenTrial(number=28, value=0.4048656668802813, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 798728), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 801949), params={'x': 2.63629055224817}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE), FrozenTrial(number=29, value=7.306360512697975, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 802236), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 805616), params={'x': -0.7030280266208813}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE), FrozenTrial(number=30, value=38.37615056296512, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 805917), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 809309), params={'x': 8.194848711870623}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE), FrozenTrial(number=31, value=1.7484037181145708, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 809608), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 813009), params={'x': 3.322272180042585}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE), FrozenTrial(number=32, value=0.2060706209667366, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 813315), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 816720), params={'x': 1.5460499796599447}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE), FrozenTrial(number=33, value=12.415806211910033, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 817024), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 820255), params={'x': 5.523606988855317}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE), FrozenTrial(number=34, value=3.7897222701357567, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 820509), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 823822), params={'x': 0.053279098037996464}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE), FrozenTrial(number=35, value=0.05723598115156259, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 824107), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 827345), params={'x': 1.760759574587482}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE), FrozenTrial(number=36, value=9.51146545516059, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 827651), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 830811), params={'x': 5.084066383066452}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE), FrozenTrial(number=37, value=0.6823697947254954, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 831119), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 834150), params={'x': 2.8260567745170397}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE), FrozenTrial(number=38, value=20.941404634893978, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 834421), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 837623), params={'x': -2.57617795052749}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE), FrozenTrial(number=39, value=5.360217744846539, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 837989), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 841430), params={'x': -0.3152144058049005}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE), FrozenTrial(number=40, value=25.186264394335122, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 841739), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 844734), params={'x': 7.018591873656905}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE), FrozenTrial(number=41, value=0.5939219916529807, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 845041), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 848417), params={'x': 1.2293366547882398}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE), FrozenTrial(number=42, value=0.016759064428153916, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 848734), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 851940), params={'x': 2.1294568052601095}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE), FrozenTrial(number=43, value=0.24399763532677102, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 852251), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 855690), params={'x': 2.4939611678328277}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE), FrozenTrial(number=44, value=3.808029839728935, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 855998), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 859503), params={'x': 3.9514173924942186}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE), FrozenTrial(number=45, value=2.1402064597508224, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 859817), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 863219), params={'x': 0.5370555513790614}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE), FrozenTrial(number=46, value=2.495705646351366, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 863475), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 866488), params={'x': 3.5797802525514}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE), FrozenTrial(number=47, value=0.12391133865990965, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 866777), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 870150), params={'x': 2.3520104240784776}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE), FrozenTrial(number=48, value=35.083735651440236, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 870442), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 873840), params={'x': -3.9231525095543702}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE), FrozenTrial(number=49, value=0.9257074131620384, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 874159), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 877607), params={'x': 1.0378631006129957}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE), FrozenTrial(number=50, value=11.309314837600713, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 877915), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 881309), params={'x': -1.3629324759204895}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE), FrozenTrial(number=51, value=2.1602300535032475, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 881760), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 885257), params={'x': 3.469772109377249}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE), FrozenTrial(number=52, value=0.01605983225882903, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 885551), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 888933), params={'x': 2.1267273934823447}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE), FrozenTrial(number=53, value=0.03252368306114132, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 889248), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 892654), params={'x': 2.180343236804548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE), FrozenTrial(number=54, value=4.529485282904583, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 892969), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 896251), params={'x': -0.12825874435055074}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE), FrozenTrial(number=55, value=7.504095970581945, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 896544), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 899955), params={'x': 4.739360503946486}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE), FrozenTrial(number=56, value=1.3728618449224683, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 900267), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 903695), params={'x': 3.1716918728584185}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE), FrozenTrial(number=57, value=14.872701673963075, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 904010), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 907407), params={'x': 5.856514186926203}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE), FrozenTrial(number=58, value=0.6700681771333539, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 907717), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 911104), params={'x': 1.1814230780596402}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE), FrozenTrial(number=59, value=4.560957931442746, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 911396), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 914551), params={'x': 4.1356399348773065}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE), FrozenTrial(number=60, value=0.0009416351200777964, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 914896), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 918411), params={'x': 2.0306860737155765}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE), FrozenTrial(number=61, value=0.014019313821028393, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 918725), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 922009), params={'x': 2.1184031833230357}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE), FrozenTrial(number=62, value=2.5497407091861928, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 922269), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 925684), params={'x': 0.40320924689983473}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE), FrozenTrial(number=63, value=0.1748122728798292, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 925977), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 929154), params={'x': 1.5818944237637709}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE), FrozenTrial(number=64, value=129.3929957707187, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 929471), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 932867), params={'x': -9.375104209224578}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE), FrozenTrial(number=65, value=0.9089481099015729, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 933163), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 936572), params={'x': 2.953387701778019}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE), FrozenTrial(number=66, value=8.203837016184718, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 936867), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 940147), params={'x': -0.8642341063859844}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE), FrozenTrial(number=67, value=1.3257727100084467, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 940473), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 943933), params={'x': 0.8485779618191918}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE), FrozenTrial(number=68, value=15.143872574857221, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 944231), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 947713), params={'x': -1.8915128902339795}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE), FrozenTrial(number=69, value=0.0014167156742555067, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 948006), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 951459), params={'x': 1.9623607163424235}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE), FrozenTrial(number=70, value=19.30238887609378, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 951774), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 955186), params={'x': 6.393448403713623}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE), FrozenTrial(number=71, value=0.019810281818791773, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 955466), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 958720), params={'x': 2.1407490029051424}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE), FrozenTrial(number=72, value=0.16871169720432258, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 959001), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 962498), params={'x': 1.5892547051951507}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE), FrozenTrial(number=73, value=2.930255013950916, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 962817), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 966297), params={'x': 3.7117987656120435}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE), FrozenTrial(number=74, value=5.825968551570679, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 966615), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 970093), params={'x': -0.4137043214881726}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE), FrozenTrial(number=75, value=0.665872785501191, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 970410), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 973540), params={'x': 2.816010285168754}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE), FrozenTrial(number=76, value=3.2145131330130496, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 973804), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 976868), params={'x': 0.20709366306740673}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE), FrozenTrial(number=77, value=6.483468422998057, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 977185), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 980688), params={'x': 4.5462655837516355}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE), FrozenTrial(number=78, value=0.01308281676473269, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 981007), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 984315), params={'x': 1.8856198585211021}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE), FrozenTrial(number=79, value=0.6427022123279538, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 984644), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 988152), params={'x': 1.1983128962444551}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE), FrozenTrial(number=80, value=1.3731589723903712, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 988491), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 991853), params={'x': 3.171818660198911}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE), FrozenTrial(number=81, value=0.06937132462989902, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 992174), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 995595), params={'x': 2.263384366715071}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE), FrozenTrial(number=82, value=0.02938787745917516, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 995919), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 22, 999487), params={'x': 1.8285710716968249}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE), FrozenTrial(number=83, value=1.8331801077744525, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 22, 999810), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 3348), params={'x': 0.6460501826971383}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE), FrozenTrial(number=84, value=0.04395233717093184, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 3673), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 7194), params={'x': 1.7903518729610688}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE), FrozenTrial(number=85, value=0.3840790907615754, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 7518), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 11081), params={'x': 2.6197411481913844}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE), FrozenTrial(number=86, value=0.4819899637306362, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 11403), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 14855), params={'x': 1.3057450297400557}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE), FrozenTrial(number=87, value=3.53356400346849, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 15197), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 18654), params={'x': 3.879777647347816}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE), FrozenTrial(number=88, value=1.6345958707059454, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 18981), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 22458), params={'x': 3.2785131484290435}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE), FrozenTrial(number=89, value=0.24341421286390458, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 22761), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 26207), params={'x': 2.493370259403528}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE), FrozenTrial(number=90, value=1.5877008040572769, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 26539), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 30093), params={'x': 0.739959999024921}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE), FrozenTrial(number=91, value=0.0007840407526423287, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 30420), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 33829), params={'x': 1.9719992722837008}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE), FrozenTrial(number=92, value=0.004876335302296941, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 34157), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 37731), params={'x': 1.9301692381375017}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE), FrozenTrial(number=93, value=3.4096460779709172, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 38061), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 41612), params={'x': 0.15347730098682044}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE), FrozenTrial(number=94, value=0.008049850308438236, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 41944), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 45452), params={'x': 1.910279041977706}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE), FrozenTrial(number=95, value=0.28787193144168016, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 45792), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 49249), params={'x': 1.4634630195022154}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE), FrozenTrial(number=96, value=0.8185657919358443, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 49578), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 53060), params={'x': 2.9047462583154706}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE), FrozenTrial(number=97, value=1.164243350865885, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 53369), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 56828), params={'x': 0.9209989106280361}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE), FrozenTrial(number=98, value=8.886738769324621, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 57158), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 60726), params={'x': 4.981063362178775}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE), FrozenTrial(number=99, value=5.665518147496434, datetime_start=datetime.datetime(2020, 11, 4, 4, 32, 23, 61057), datetime_complete=datetime.datetime(2020, 11, 4, 4, 32, 23, 64617), params={'x': -0.3802348933448634}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

Total running time of the script: ( 0 minutes 0.763 seconds)

Gallery generated by Sphinx-Gallery