First Optimization

Quadratic Function Example

Usually, Optuna is used to optimize hyper-parameters, but as an example, let us directly optimize a quadratic function in an IPython shell.

import optuna

The objective function is what will be optimized.

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

print(study.best_params)

Out:

{'x': 2.000759398446823}

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyper-parameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the best parameter:

study.best_params

Out:

{'x': 2.000759398446823}

To get the best value:

study.best_value

Out:

5.766860010372535e-07

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=61, value=5.766860010372535e-07, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 859258), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 862250), params={'x': 2.000759398446823}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, value=101.3105598792585, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 672725), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 672927), params={'x': -8.065314693503552}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE), FrozenTrial(number=1, value=0.449440705439096, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 673661), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 673775), params={'x': 2.6704033900862196}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE), FrozenTrial(number=2, value=9.86324655060735, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 674271), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 674368), params={'x': 5.140580607245633}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE), FrozenTrial(number=3, value=84.45516557043152, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 674843), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 674941), params={'x': -7.189949160383398}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE), FrozenTrial(number=4, value=2.914489962150087, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 675504), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 675601), params={'x': 0.2928122651125662}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE), FrozenTrial(number=5, value=4.425042693065823, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 676168), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 676267), params={'x': -0.10357854454399273}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE), FrozenTrial(number=6, value=15.289873332041688, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 676854), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 676954), params={'x': -1.9102267622276958}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE), FrozenTrial(number=7, value=17.657532095942372, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 677517), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 677617), params={'x': 6.202086635939622}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE), FrozenTrial(number=8, value=15.626508281590864, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 678219), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 678321), params={'x': -1.9530378548137968}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE), FrozenTrial(number=9, value=14.296745542264278, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 678883), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 678984), params={'x': 5.781103746561879}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE), FrozenTrial(number=10, value=52.46155336743144, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 679500), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 682703), params={'x': 9.243034817494076}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE), FrozenTrial(number=11, value=0.1651447064652934, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 683306), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 686262), params={'x': 2.4063800025410864}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE), FrozenTrial(number=12, value=0.1174099710327117, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 686927), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 689505), params={'x': 2.3426513841103107}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE), FrozenTrial(number=13, value=0.6366736320856554, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 690196), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 693902), params={'x': 2.7979183116620745}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE), FrozenTrial(number=14, value=41.027574501055824, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 694538), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 697414), params={'x': -4.4052770823014225}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE), FrozenTrial(number=15, value=57.852807770495716, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 698072), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 700763), params={'x': 9.606103323679985}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE), FrozenTrial(number=16, value=1.5535705298995757, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 701493), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 704577), params={'x': 3.2464230942579553}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE), FrozenTrial(number=17, value=33.76968292983032, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 705202), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 708072), params={'x': 7.8111688092698115}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE), FrozenTrial(number=18, value=1.498766354580575, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 708741), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 711199), params={'x': 0.7757588658354209}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE), FrozenTrial(number=19, value=46.21140868716865, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 711921), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 714988), params={'x': -4.7978973725092855}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE), FrozenTrial(number=20, value=5.907247253099437, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 715657), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 718750), params={'x': 4.430482925901648}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE), FrozenTrial(number=21, value=0.052761214430752545, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 719442), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 722359), params={'x': 2.2296980940947324}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE), FrozenTrial(number=22, value=0.044651107244000995, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 723028), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 725753), params={'x': 1.7886919139171409}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE), FrozenTrial(number=23, value=11.731784221415205, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 726445), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 729160), params={'x': -1.4251692252230699}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE), FrozenTrial(number=24, value=0.6689807074647734, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 729803), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 732688), params={'x': 1.1820875918139073}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE), FrozenTrial(number=25, value=4.772730997843733, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 733460), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 736439), params={'x': 4.184658096326227}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE), FrozenTrial(number=26, value=28.099232168988173, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 737058), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 739804), params={'x': 7.300870887787041}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE), FrozenTrial(number=27, value=32.44337917609542, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 740398), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 743167), params={'x': -3.695908985938541}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE), FrozenTrial(number=28, value=0.45541746431802416, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 743783), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 746422), params={'x': 1.325153747644677}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE), FrozenTrial(number=29, value=5.668451911159803, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 747044), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 750202), params={'x': -0.3808510896651649}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE), FrozenTrial(number=30, value=2.8743094757033374, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 750848), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 753559), params={'x': 3.69537885904695}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE), FrozenTrial(number=31, value=0.0014153354299023568, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 754186), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 757069), params={'x': 2.037620944032578}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE), FrozenTrial(number=32, value=0.042796220112888965, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 757713), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 760698), params={'x': 2.206872473067078}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE), FrozenTrial(number=33, value=0.15289708111714143, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 761379), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 764185), params={'x': 1.6089794364523249}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE), FrozenTrial(number=34, value=8.3287110609188, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 764880), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 767575), params={'x': -0.8859506338326026}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE), FrozenTrial(number=35, value=24.697964081836037, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 768188), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 771062), params={'x': -2.969704627222431}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE), FrozenTrial(number=36, value=0.019868766825215683, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 771734), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 774694), params={'x': 1.8590433867276328}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE), FrozenTrial(number=37, value=9.841453421951272, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 775365), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 778303), params={'x': 5.137109086715231}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE), FrozenTrial(number=38, value=3.0308580208571443, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 778985), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 782111), params={'x': 0.25906403884084694}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE), FrozenTrial(number=39, value=6.32987568396814, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 783075), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 785996), params={'x': 4.515924419367192}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE), FrozenTrial(number=40, value=1.4107038096008262, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 786658), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 789615), params={'x': 3.1877305290346065}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE), FrozenTrial(number=41, value=0.005149442410297911, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 790294), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 793185), params={'x': 2.0717596154553375}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE), FrozenTrial(number=42, value=2.85882792836227, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 793783), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 796695), params={'x': 0.3091931132260344}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE), FrozenTrial(number=43, value=0.16224499318968533, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 797246), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 800148), params={'x': 1.5972035337919592}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE), FrozenTrial(number=44, value=15.698915760269912, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 800750), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 803479), params={'x': 5.962185730158281}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE), FrozenTrial(number=45, value=0.008845861784295155, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 804042), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 807068), params={'x': 1.9059475583289027}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE), FrozenTrial(number=46, value=2.2329168427654844, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 807741), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 810777), params={'x': 3.494294764350556}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE), FrozenTrial(number=47, value=126.09099610789094, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 811479), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 814177), params={'x': -9.229024717574138}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE), FrozenTrial(number=48, value=7.552389014953103, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 814697), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 817281), params={'x': -0.7481610242038408}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE), FrozenTrial(number=49, value=18.2703426332367, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 817818), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 820555), params={'x': -2.2743821346759225}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE), FrozenTrial(number=50, value=23.77864250133567, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 821148), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 823865), params={'x': 6.876334945564719}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE), FrozenTrial(number=51, value=0.0059166420659145765, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 824462), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 827392), params={'x': 2.0769197118163776}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE), FrozenTrial(number=52, value=0.45269451566342966, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 828081), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 830883), params={'x': 2.6728257691731416}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE), FrozenTrial(number=53, value=1.7507178333951159, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 831407), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 833973), params={'x': 0.6768530567638695}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE), FrozenTrial(number=54, value=9.679817986479323, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 834493), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 837085), params={'x': 5.111240586402685}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE), FrozenTrial(number=55, value=0.035180694436743626, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 837746), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 840685), params={'x': 2.1875651738376387}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE), FrozenTrial(number=56, value=2.8130959571235308, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 841381), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 844423), params={'x': 3.677228653798739}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE), FrozenTrial(number=57, value=0.5545709365852781, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 844946), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 847856), params={'x': 2.744695197100987}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE), FrozenTrial(number=58, value=1.0433596018982385, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 848484), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 851706), params={'x': 0.9785502450447019}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE), FrozenTrial(number=59, value=4.883294865136924, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 852372), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 855143), params={'x': -0.20981783528347053}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE), FrozenTrial(number=60, value=10.838597649306646, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 855765), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 858617), params={'x': -1.2922025528977779}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE), FrozenTrial(number=61, value=5.766860010372535e-07, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 859258), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 862250), params={'x': 2.000759398446823}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE), FrozenTrial(number=62, value=0.00020714113606811423, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 862846), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 865812), params={'x': 2.0143923985516006}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE), FrozenTrial(number=63, value=4.770021822349454, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 866517), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 869600), params={'x': 4.184037962662154}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE), FrozenTrial(number=64, value=1.539295955166531, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 870283), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 873462), params={'x': 0.7593163355768195}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE), FrozenTrial(number=65, value=1.0672923105489198, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 874225), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 877172), params={'x': 3.0330984031295953}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE), FrozenTrial(number=66, value=0.1147878556185434, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 877837), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 880892), params={'x': 1.6611964350563244}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE), FrozenTrial(number=67, value=1.1361457053613755e-06, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 881527), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 884617), params={'x': 1.998934098641824}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE), FrozenTrial(number=68, value=3.6246461529732192, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 885281), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 888287), params={'x': 0.09614965058352887}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE), FrozenTrial(number=69, value=3.724216739192673, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 889061), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 892299), params={'x': 3.929822981310118}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE), FrozenTrial(number=70, value=7.447001245292302, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 892975), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 896042), params={'x': 4.728919428142264}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE), FrozenTrial(number=71, value=0.005146703754407063, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 896712), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 899882), params={'x': 1.9282594692352568}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE), FrozenTrial(number=72, value=0.08312811414414513, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 900626), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 903749), params={'x': 2.288319465427059}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE), FrozenTrial(number=73, value=0.4842851724589894, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 904492), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 907828), params={'x': 1.3040939916490235}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE), FrozenTrial(number=74, value=0.8487633862069828, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 908489), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 911495), params={'x': 2.921283553639694}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE), FrozenTrial(number=75, value=1.0724541780781007, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 912163), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 915175), params={'x': 0.9644063644082681}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE), FrozenTrial(number=76, value=0.029934028497446395, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 915912), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 918841), params={'x': 2.173014532619218}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE), FrozenTrial(number=77, value=2.1546355054214636, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 919518), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 922855), params={'x': 0.532132327005781}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE), FrozenTrial(number=78, value=6.639602131844719, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 923476), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 926456), params={'x': -0.5767425427940447}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE), FrozenTrial(number=79, value=1.7404365752917346, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 927055), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 930052), params={'x': 3.319256068885694}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE), FrozenTrial(number=80, value=0.0011354675131661218, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 930722), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 933852), params={'x': 1.9663033011532862}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE), FrozenTrial(number=81, value=0.08161608085214879, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 934505), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 937798), params={'x': 1.7143147171236348}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE), FrozenTrial(number=82, value=0.5929935779145988, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 938462), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 941418), params={'x': 1.2299392375178444}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE), FrozenTrial(number=83, value=0.2506133431959023, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 942112), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 944844), params={'x': 2.500612967466787}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE), FrozenTrial(number=84, value=0.0004949405982997249, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 945536), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 948647), params={'x': 2.0222472604672963}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE), FrozenTrial(number=85, value=2.425010486246729, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 949271), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 952541), params={'x': 3.557244517167015}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE), FrozenTrial(number=86, value=0.6022298074457408, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 953225), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 956684), params={'x': 2.7760346689715227}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE), FrozenTrial(number=87, value=4.480313229298921, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 957285), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 960110), params={'x': -0.11667504102517379}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE), FrozenTrial(number=88, value=0.3963287668408014, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 960706), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 963522), params={'x': 1.3704535228906436}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE), FrozenTrial(number=89, value=2.3356363221735705, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 964120), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 967282), params={'x': 0.4717211242140489}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE), FrozenTrial(number=90, value=5.429473673384479e-05, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 968060), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 970897), params={'x': 1.9926315037671283}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE), FrozenTrial(number=91, value=0.0009411822415126626, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 971594), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 974710), params={'x': 2.0306786936083117}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE), FrozenTrial(number=92, value=1.3958693458674427, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 975397), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 978682), params={'x': 3.1814691472346803}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE), FrozenTrial(number=93, value=0.3285433952668455, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 979375), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 982888), params={'x': 2.5731870508541217}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE), FrozenTrial(number=94, value=0.014433851569801827, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 983618), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 986874), params={'x': 1.8798590345893549}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE), FrozenTrial(number=95, value=4.022246770539913, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 987564), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 990782), params={'x': 4.005553980958855}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE), FrozenTrial(number=96, value=0.4077730562057798, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 991380), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 994404), params={'x': 1.3614288949492157}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE), FrozenTrial(number=97, value=1.1883220241528745, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 995039), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 9, 998103), params={'x': 0.9098981588159414}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE), FrozenTrial(number=98, value=69.9329997177723, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 9, 998867), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 10, 2066), params={'x': -6.362595274062491}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE), FrozenTrial(number=99, value=1.2046985560755772, datetime_start=datetime.datetime(2020, 9, 18, 8, 46, 10, 2663), datetime_complete=datetime.datetime(2020, 9, 18, 8, 46, 10, 5645), params={'x': 3.0975876074717577}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

Total running time of the script: ( 0 minutes 0.743 seconds)

Gallery generated by Sphinx-Gallery