First Optimization

Quadratic Function Example

Usually, Optuna is used to optimize hyper-parameters, but as an example, let us directly optimize a quadratic function in an IPython shell.

import optuna

The objective function is what will be optimized.

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

print(study.best_params)

Out:

{'x': 2.007349008829681}

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyper-parameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the best parameter:

study.best_params

Out:

{'x': 2.007349008829681}

To get the best value:

study.best_value

Out:

5.4007930778729834e-05

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=54, value=5.4007930778729834e-05, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 142234), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 145624), params={'x': 2.007349008829681}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, value=115.35868359799473, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 971466), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 971675), params={'x': -8.740515983787498}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE), FrozenTrial(number=1, value=80.69610858156149, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 972068), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 972296), params={'x': -6.983101278598694}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE), FrozenTrial(number=2, value=85.89782395415705, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 972642), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 972831), params={'x': -7.268107895042927}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE), FrozenTrial(number=3, value=7.288716381775051, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 973154), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 973348), params={'x': 4.699762282456559}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE), FrozenTrial(number=4, value=57.69843403649008, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 973669), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 973862), params={'x': -5.595948527767291}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE), FrozenTrial(number=5, value=24.976150848875275, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 974208), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 974383), params={'x': -2.997614515834057}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE), FrozenTrial(number=6, value=0.038653749772173285, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 974726), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 974899), params={'x': 2.196605569026346}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE), FrozenTrial(number=7, value=2.2776590806065715, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 975242), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 975417), params={'x': 0.4908084678853477}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE), FrozenTrial(number=8, value=102.02717901268842, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 975751), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 975939), params={'x': -8.100850410370823}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE), FrozenTrial(number=9, value=9.534197871505668, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 976274), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 976487), params={'x': 5.087749645211812}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE), FrozenTrial(number=10, value=58.11691739507765, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 976836), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 980431), params={'x': 9.623445244446742}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE), FrozenTrial(number=11, value=0.4448514071770874, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 980791), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 984089), params={'x': 1.333028181122255}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE), FrozenTrial(number=12, value=0.9489107969363859, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 984461), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 987686), params={'x': 1.0258794751487956}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE), FrozenTrial(number=13, value=10.302482765486877, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 988023), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 991299), params={'x': 5.209748084427636}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE), FrozenTrial(number=14, value=17.127602045266684, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 991683), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 995343), params={'x': -2.138550717976848}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE), FrozenTrial(number=15, value=1.7785787840373846, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 995679), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 56, 998959), params={'x': 3.3336336768533497}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE), FrozenTrial(number=16, value=46.298215197084545, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 56, 999292), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 2587), params={'x': 8.804279182770541}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE), FrozenTrial(number=17, value=0.4253723247619816, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 2944), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 6211), params={'x': 2.652205738062754}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE), FrozenTrial(number=18, value=16.987362356895066, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 6566), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 9844), params={'x': -2.1215728013581256}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE), FrozenTrial(number=19, value=0.7750964828953515, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 10203), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 13472), params={'x': 2.880395639979749}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE), FrozenTrial(number=20, value=27.770062626933537, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 13828), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 17085), params={'x': 7.269730792643353}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE), FrozenTrial(number=21, value=1.2767176934225606, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 17446), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 20725), params={'x': 0.8700806695066411}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE), FrozenTrial(number=22, value=0.7529644792428944, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 21085), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 24319), params={'x': 2.8677352587298124}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE), FrozenTrial(number=23, value=26.521306244875927, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 24699), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 27952), params={'x': 7.149884100140111}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE), FrozenTrial(number=24, value=6.9526235433418, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 28314), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 31614), params={'x': -0.6367828017001704}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE), FrozenTrial(number=25, value=0.007605709174528642, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 32046), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 35571), params={'x': 1.9127892829147206}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE), FrozenTrial(number=26, value=6.89460410811278, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 36020), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 39317), params={'x': -0.6257578159671886}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE), FrozenTrial(number=27, value=0.18995968899873011, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 39681), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 43021), params={'x': 2.43584365201151}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE), FrozenTrial(number=28, value=37.7217756229094, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 43384), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 46794), params={'x': -4.141805567006285}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE), FrozenTrial(number=29, value=142.7933762545482, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 47221), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 50610), params={'x': -9.94961824723067}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE), FrozenTrial(number=30, value=19.426534053850624, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 50978), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 54373), params={'x': 6.407554203166494}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE), FrozenTrial(number=31, value=0.4944213151474354, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 54740), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 58137), params={'x': 2.7031509902911575}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE), FrozenTrial(number=32, value=4.304417307550129, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 58506), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 61892), params={'x': 4.074708969361758}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE), FrozenTrial(number=33, value=0.20186842411651512, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 62326), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 65777), params={'x': 2.449297700991798}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE), FrozenTrial(number=34, value=0.11211560085905879, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 66200), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 69595), params={'x': 1.6651633221120201}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE), FrozenTrial(number=35, value=7.057894706072463, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 69965), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 73357), params={'x': -0.6566698526675201}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE), FrozenTrial(number=36, value=0.146098865104252, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 73726), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 77083), params={'x': 1.617771187501188}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE), FrozenTrial(number=37, value=0.07622455332989723, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 77453), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 80833), params={'x': 1.7239120550804559}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE), FrozenTrial(number=38, value=4.12973882169398, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 81208), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 84578), params={'x': 4.032175883552893}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE), FrozenTrial(number=39, value=13.817272793288295, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 84923), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 88738), params={'x': -1.7171592370099367}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE), FrozenTrial(number=40, value=3.640747422143335, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 89085), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 92509), params={'x': 0.09192572939538457}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE), FrozenTrial(number=41, value=0.010988651967577404, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 92855), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 96321), params={'x': 1.8951732287648932}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE), FrozenTrial(number=42, value=0.14692526313280416, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 96741), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 100181), params={'x': 1.6166916865853231}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE), FrozenTrial(number=43, value=15.652634371317928, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 100550), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 103933), params={'x': 5.956341033242449}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE), FrozenTrial(number=44, value=3.896926179559272, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 104278), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 107734), params={'x': 3.9740633676656056}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE), FrozenTrial(number=45, value=2.251135236754777, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 108080), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 111495), params={'x': 0.49962163546831406}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE), FrozenTrial(number=46, value=0.025313948456615862, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 111843), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 115296), params={'x': 1.8408964222381663}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE), FrozenTrial(number=47, value=7.086268428234422, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 115648), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 119076), params={'x': 4.662004588319566}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE), FrozenTrial(number=48, value=9.383622462439314, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 119422), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 122860), params={'x': -1.0632698970935153}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE), FrozenTrial(number=49, value=27.98857757117444, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 123212), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 126636), params={'x': -3.290423193958536}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE), FrozenTrial(number=50, value=2.7205615153113887, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 126991), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 130443), params={'x': 0.35058752420403083}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE), FrozenTrial(number=51, value=0.011038180067781627, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 130801), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 134274), params={'x': 1.8949372565188705}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE), FrozenTrial(number=52, value=0.04076565323187306, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 134625), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 138073), params={'x': 1.7980949400538138}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE), FrozenTrial(number=53, value=2.0980225623467166, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 138435), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 141863), params={'x': 3.44845523311793}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE), FrozenTrial(number=54, value=5.4007930778729834e-05, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 142234), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 145624), params={'x': 2.007349008829681}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE), FrozenTrial(number=55, value=9.861012545809858, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 146006), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 149412), params={'x': 5.140224919621182}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE), FrozenTrial(number=56, value=1.2897828943028955, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 149786), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 153191), params={'x': 0.864313910315489}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE), FrozenTrial(number=57, value=1.9380941980229127, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 153567), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 157380), params={'x': 3.3921545165759843}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE), FrozenTrial(number=58, value=4.011331126562202, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 157769), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 161171), params={'x': -0.0028307783140844034}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE), FrozenTrial(number=59, value=0.06164900082830564, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 161552), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 165042), params={'x': 2.2482921682782315}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE), FrozenTrial(number=60, value=7.271786025640625, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 165423), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 168873), params={'x': 4.696624932325707}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE), FrozenTrial(number=61, value=6.726898865597165e-05, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 169255), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 172676), params={'x': 2.008201767410502}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE), FrozenTrial(number=62, value=0.8831258008194665, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 173062), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 176476), params={'x': 1.0602522674571295}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE), FrozenTrial(number=63, value=1.820435322367306, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 176854), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 180271), params={'x': 3.3492350878802797}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE), FrozenTrial(number=64, value=0.07567399704088205, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 180648), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 184053), params={'x': 2.2750890711040372}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE), FrozenTrial(number=65, value=4.764727317600995, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 184412), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 187829), params={'x': -0.18282553530990997}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE), FrozenTrial(number=66, value=12.41261982198241, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 188195), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 191629), params={'x': -1.5231548109588389}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE), FrozenTrial(number=67, value=1.6773405555035814, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 191987), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 195407), params={'x': 0.7048781696289799}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE), FrozenTrial(number=68, value=0.6345367460400834, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 195767), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 199200), params={'x': 1.203421851893938}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE), FrozenTrial(number=69, value=1.3205379003343043, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 199560), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 202989), params={'x': 3.1491465965377543}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE), FrozenTrial(number=70, value=0.008885953945958694, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 203349), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 206784), params={'x': 2.094265337987824}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE), FrozenTrial(number=71, value=0.1915323944151887, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 207144), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 210584), params={'x': 2.437644141301113}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE), FrozenTrial(number=72, value=4.126023474750347, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 210943), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 214372), params={'x': 4.031261547598031}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE), FrozenTrial(number=73, value=0.012187679060030626, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 214738), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 218199), params={'x': 1.889602178191639}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE), FrozenTrial(number=74, value=0.20269285749140598, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 218560), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 222014), params={'x': 1.5497857648947493}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE), FrozenTrial(number=75, value=0.025510658253676134, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 222376), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 225834), params={'x': 2.1597205630270446}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE), FrozenTrial(number=76, value=0.8018291657237827, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 226200), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 229653), params={'x': 2.8954491418968376}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE), FrozenTrial(number=77, value=0.7125612218713069, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 230016), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 233455), params={'x': 1.1558665852655121}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE), FrozenTrial(number=78, value=6.217456179020683, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 233817), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 237299), params={'x': -0.4934827408708251}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE), FrozenTrial(number=79, value=0.006485294221857291, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 237659), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 241142), params={'x': 2.0805313244759907}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE), FrozenTrial(number=80, value=2.706178928311269, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 241502), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 244958), params={'x': 0.35495321394457924}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE), FrozenTrial(number=81, value=0.009207036839518042, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 245320), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 248808), params={'x': 1.904046694483629}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE), FrozenTrial(number=82, value=2.284878572785951, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 249174), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 252647), params={'x': 3.511581480696939}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE), FrozenTrial(number=83, value=0.7667041782810153, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 253010), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 256531), params={'x': 2.8756164561501887}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE), FrozenTrial(number=84, value=0.03971338023254167, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 256898), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 260354), params={'x': 2.1992821623541396}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE), FrozenTrial(number=85, value=0.5031040471968893, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 260735), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 264205), params={'x': 1.2907017219836994}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE), FrozenTrial(number=86, value=0.011505064490723823, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 264588), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 268064), params={'x': 1.8927383363418047}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE), FrozenTrial(number=87, value=6.154759838448609, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 268428), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 271897), params={'x': 4.480878843968123}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE), FrozenTrial(number=88, value=15.524570895827814, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 272263), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 275734), params={'x': 5.940123208204004}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE), FrozenTrial(number=89, value=3.3894835202599065, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 276104), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 279595), params={'x': 3.8410550019648806}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE), FrozenTrial(number=90, value=1.7247630773643812, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 279963), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 283461), params={'x': 0.6866976443467478}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE), FrozenTrial(number=91, value=0.00015521880530894354, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 283825), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 287351), params={'x': 2.0124586839316576}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE), FrozenTrial(number=92, value=0.30803772639153776, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 287722), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 291230), params={'x': 2.5550114650991795}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE), FrozenTrial(number=93, value=0.3693759613227988, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 291600), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 295101), params={'x': 1.3922369200726332}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE), FrozenTrial(number=94, value=1.3324832643141309, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 295474), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 299009), params={'x': 3.154332389008526}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE), FrozenTrial(number=95, value=0.6310517902917445, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 299377), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 302865), params={'x': 2.7943876826158274}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE), FrozenTrial(number=96, value=4.374261347557365, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 303241), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 306756), params={'x': -0.0914734871753371}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE), FrozenTrial(number=97, value=0.00212826273069274, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 307127), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 310627), params={'x': 1.9538669020041712}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE), FrozenTrial(number=98, value=5.587635129515223, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 310998), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 314513), params={'x': 4.36381791378169}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE), FrozenTrial(number=99, value=2.8178419150102356, datetime_start=datetime.datetime(2020, 10, 5, 4, 41, 57, 314886), datetime_complete=datetime.datetime(2020, 10, 5, 4, 41, 57, 318421), params={'x': 3.6786428789382914}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

Total running time of the script: ( 0 minutes 0.783 seconds)

Gallery generated by Sphinx-Gallery