First Optimization

Quadratic Function Example

Usually, Optuna is used to optimize hyper-parameters, but as an example, let us directly optimize a quadratic function in an IPython shell.

import optuna

The objective function is what will be optimized.

def objective(trial):
    x = trial.suggest_uniform('x', -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

print(study.best_params)

Out:

{'x': 2.014785252062935}

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyper-parameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the best parameter:

study.best_params

Out:

{'x': 2.014785252062935}

To get the best value:

study.best_value

Out:

0.0002186036785645176

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=97, value=0.0002186036785645176, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 818707), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 821790), params={'x': 2.014785252062935}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, value=19.62482979388249, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 546054), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 546240), params={'x': -2.429992076051885}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE), FrozenTrial(number=1, value=0.10597856096821312, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 546557), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 546692), params={'x': 2.325543485525687}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE), FrozenTrial(number=2, value=1.6370312490105587, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 546986), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 547126), params={'x': 0.7205347800699862}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE), FrozenTrial(number=3, value=5.688281835867098, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 547360), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 547473), params={'x': 4.385011915246357}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE), FrozenTrial(number=4, value=59.981661193357276, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 547718), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 547840), params={'x': 9.744782837068918}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE), FrozenTrial(number=5, value=18.487942318754463, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 548077), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 548201), params={'x': -2.299760728081792}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE), FrozenTrial(number=6, value=13.32552995855641, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 548431), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 548550), params={'x': 5.6504150392190216}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE), FrozenTrial(number=7, value=22.504215302333098, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 548789), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 548914), params={'x': 6.743860801323443}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE), FrozenTrial(number=8, value=9.217910492085108, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 549142), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 549275), params={'x': -1.0361011992496412}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE), FrozenTrial(number=9, value=24.326104819346014, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 549510), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 549630), params={'x': 6.932150121331063}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE), FrozenTrial(number=10, value=79.76117409224267, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 549857), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 552884), params={'x': -6.930911156888902}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE), FrozenTrial(number=11, value=0.008315288058716714, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 553133), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 556316), params={'x': 2.0911882013130905}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE), FrozenTrial(number=12, value=0.20057502527662305, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 556665), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 559791), params={'x': 1.5521439681363853}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE), FrozenTrial(number=13, value=0.45409351719793395, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 560050), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 562348), params={'x': 2.67386461340386}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE), FrozenTrial(number=14, value=35.30969167096243, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 562627), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 564901), params={'x': -3.9421958627230076}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE), FrozenTrial(number=15, value=0.611911653589387, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 565146), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 567679), params={'x': 2.7822478210831827}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE), FrozenTrial(number=16, value=53.56668610741811, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 567996), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 570931), params={'x': -5.318926567975533}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE), FrozenTrial(number=17, value=40.923043578294234, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 571210), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 573602), params={'x': 8.397112128006999}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE), FrozenTrial(number=18, value=124.16487092524956, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 573861), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 576141), params={'x': -9.14292918963634}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE), FrozenTrial(number=19, value=6.850726774285624, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 576387), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 578864), params={'x': 4.617389305068244}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE), FrozenTrial(number=20, value=5.872004163876423, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 579110), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 581494), params={'x': -0.42322185609911145}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE), FrozenTrial(number=21, value=0.08532425263540877, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 581740), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 584394), params={'x': 1.7078968458995885}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE), FrozenTrial(number=22, value=0.15177009764794747, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 584711), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 587515), params={'x': 2.389576818673734}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE), FrozenTrial(number=23, value=4.671460546849846, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 587796), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 590133), params={'x': 4.161356182319297}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE), FrozenTrial(number=24, value=2.401897683385193, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 590379), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 592684), params={'x': 0.4501943078614039}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE), FrozenTrial(number=25, value=2.000211584996334, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 592931), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 595265), params={'x': 3.414288366987558}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE), FrozenTrial(number=26, value=11.400216702395879, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 595548), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 598108), params={'x': -1.376420693929576}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE), FrozenTrial(number=27, value=0.4990443599298654, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 598427), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 601491), params={'x': 1.2935692815782531}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE), FrozenTrial(number=28, value=20.9853300769071, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 601746), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 604133), params={'x': 6.580974795489176}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE), FrozenTrial(number=29, value=35.4874437253311, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 604387), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 607012), params={'x': -3.957133851554042}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE), FrozenTrial(number=30, value=11.154490861382127, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 607266), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 610293), params={'x': 5.33983395715747}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE), FrozenTrial(number=31, value=0.1667571307282857, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 610637), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 613433), params={'x': 2.408359070828953}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE), FrozenTrial(number=32, value=0.0814265563537211, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 613718), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 616965), params={'x': 1.714646611455688}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE), FrozenTrial(number=33, value=2.7958886533734235, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 617266), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 620153), params={'x': 0.3279088980042315}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE), FrozenTrial(number=34, value=0.2869010654254136, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 620437), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 623343), params={'x': 1.464368535814583}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE), FrozenTrial(number=35, value=5.434151475029695, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 623681), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 626776), params={'x': -0.33112665357970084}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE), FrozenTrial(number=36, value=3.0484024496019884, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 627122), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 630243), params={'x': 3.7459674824010865}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE), FrozenTrial(number=37, value=14.828375730586558, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 630592), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 633936), params={'x': -1.8507630062867488}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE), FrozenTrial(number=38, value=24.47742291324365, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 634219), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 636929), params={'x': -2.947466312492047}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE), FrozenTrial(number=39, value=0.7892236850339701, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 637226), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 639728), params={'x': 1.1116173768955349}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE), FrozenTrial(number=40, value=6.084271852283226, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 639983), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 642400), params={'x': 4.466631681521022}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE), FrozenTrial(number=41, value=0.03654191334805851, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 642657), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 645285), params={'x': 2.1911593925185433}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE), FrozenTrial(number=42, value=0.02280086455175231, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 645606), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 648576), params={'x': 2.1509995514952025}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE), FrozenTrial(number=43, value=0.011677481245499255, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 648871), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 651260), params={'x': 1.8919376048502567}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE), FrozenTrial(number=44, value=11.852176071624395, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 651516), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 653960), params={'x': 5.44269895164018}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE), FrozenTrial(number=45, value=2.4040138924962826, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 654219), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 656782), params={'x': 3.5504882755107445}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE), FrozenTrial(number=46, value=7.598424203265841, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 657041), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 659593), params={'x': -0.7565239348254971}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE), FrozenTrial(number=47, value=1.1991911109443805, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 659880), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 662971), params={'x': 3.0950758471194497}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE), FrozenTrial(number=48, value=2.01147690558948, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 663322), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 666241), params={'x': 0.5817345433278438}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE), FrozenTrial(number=49, value=0.08758345655625584, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 666496), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 668877), params={'x': 2.295945022861098}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE), FrozenTrial(number=50, value=8.876972600846395, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 669174), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 671730), params={'x': 4.979424877530292}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE), FrozenTrial(number=51, value=0.009177569258989406, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 671985), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 674400), params={'x': 1.9042003692126666}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE), FrozenTrial(number=52, value=0.02403315207994953, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 674656), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 677389), params={'x': 1.8449737051982809}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE), FrozenTrial(number=53, value=4.088566745240392, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 677718), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 680631), params={'x': -0.022020461132970848}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE), FrozenTrial(number=54, value=17.62744335897347, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 680888), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 683334), params={'x': 6.198504895671014}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE), FrozenTrial(number=55, value=0.0030167664165317923, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 683592), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 686017), params={'x': 2.0549250982387086}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE), FrozenTrial(number=56, value=0.8425722028651674, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 686273), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 688840), params={'x': 2.9179173180985134}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE), FrozenTrial(number=57, value=1.1421500687222987, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 689097), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 691514), params={'x': 0.931285787161835}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE), FrozenTrial(number=58, value=4.251914392554242, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 691820), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 695185), params={'x': 4.0620170689289266}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE), FrozenTrial(number=59, value=29.729222119335585, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 695443), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 698107), params={'x': 7.4524510194347995}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE), FrozenTrial(number=60, value=10.695313384913042, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 698363), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 701179), params={'x': -1.2703689982803228}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE), FrozenTrial(number=61, value=0.05976560538106649, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 701474), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 705913), params={'x': 2.244470050069669}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE), FrozenTrial(number=62, value=0.005685304866540179, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 706276), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 712303), params={'x': 2.07540096064733}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE), FrozenTrial(number=63, value=1.3650841483790592, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 712644), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 717072), params={'x': 3.1683681561815433}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE), FrozenTrial(number=64, value=0.06634267442655128, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 717445), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 720433), params={'x': 1.7424292826687178}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE), FrozenTrial(number=65, value=3.719561273078722, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 720698), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 723197), params={'x': 0.07138358581113335}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE), FrozenTrial(number=66, value=1.2328389122093755, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 723492), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 726475), params={'x': 0.8896672065504976}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE), FrozenTrial(number=67, value=0.036757726233703845, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 726737), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 729530), params={'x': 1.808276954348978}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE), FrozenTrial(number=68, value=3.3522907026096216, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 729788), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 732373), params={'x': 3.830926187100294}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE), FrozenTrial(number=69, value=0.6543003418810552, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 732669), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 735152), params={'x': 2.808888337090513}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE), FrozenTrial(number=70, value=7.739468075883116, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 735412), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 737846), params={'x': 4.7819899489184206}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE), FrozenTrial(number=71, value=0.07466236031110812, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 738107), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 740824), params={'x': 2.2732441404881505}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE), FrozenTrial(number=72, value=0.6137474449409213, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 741180), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 744022), params={'x': 1.216579649906309}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE), FrozenTrial(number=73, value=0.0005238738510991414, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 744282), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 746715), params={'x': 2.022888290698502}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE), FrozenTrial(number=74, value=2.6425842753726876, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 746976), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 749413), params={'x': 0.37439725782321354}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE), FrozenTrial(number=75, value=7.308596017292054, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 749674), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 752107), params={'x': -0.7034415135697043}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE), FrozenTrial(number=76, value=1.957469168941915, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 752369), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 755392), params={'x': 3.399095839798659}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE), FrozenTrial(number=77, value=0.24145029092102066, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 755751), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 759218), params={'x': 1.5086240839021303}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE), FrozenTrial(number=78, value=0.5658589716346727, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 759580), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 762196), params={'x': 2.752235981348056}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE), FrozenTrial(number=79, value=0.0015341700944712486, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 762459), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 765254), params={'x': 2.0391684834333836}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE), FrozenTrial(number=80, value=1.4034136245525186, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 765517), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 768565), params={'x': 0.8153424019774664}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE), FrozenTrial(number=81, value=0.026365402808997674, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 768831), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 771822), params={'x': 2.1623742676934916}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE), FrozenTrial(number=82, value=0.037397737460892214, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 772182), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 775181), params={'x': 1.806615053685939}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE), FrozenTrial(number=83, value=4.924009029759738, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 775545), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 778268), params={'x': -0.21901082236201286}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE), FrozenTrial(number=84, value=4.5895911208671745, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 778534), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 781403), params={'x': 4.142333102219908}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE), FrozenTrial(number=85, value=0.45267811657290374, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 781707), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 784648), params={'x': 2.6728135823338466}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE), FrozenTrial(number=86, value=0.5158005411752152, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 784915), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 787966), params={'x': 1.281807448398958}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE), FrozenTrial(number=87, value=2.7051952268730846, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 788331), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 791065), params={'x': 3.644747769985746}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE), FrozenTrial(number=88, value=1.8057111321258903, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 791405), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 794159), params={'x': 0.656232485834737}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE), FrozenTrial(number=89, value=1.5758767132402256, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 794425), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 797327), params={'x': 3.255339282122656}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE), FrozenTrial(number=90, value=0.0008008345924033004, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 797594), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 800455), params={'x': 1.9717009789497357}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE), FrozenTrial(number=91, value=0.003766908369438668, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 800726), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 803699), params={'x': 1.938624855442625}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE), FrozenTrial(number=92, value=0.30126730750339564, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 804064), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 806837), params={'x': 2.548878226479604}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE), FrozenTrial(number=93, value=0.576524466801405, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 807104), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 809663), params={'x': 1.2407079173325952}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE), FrozenTrial(number=94, value=3.3490046548183297, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 809969), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 812492), params={'x': 0.16997140601073402}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE), FrozenTrial(number=95, value=0.014473096415234699, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 812760), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 815264), params={'x': 1.8796958171332572}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE), FrozenTrial(number=96, value=1.0174980821269362, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 815531), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 818392), params={'x': 3.0087110994367694}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE), FrozenTrial(number=97, value=0.0002186036785645176, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 818707), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 821790), params={'x': 2.014785252062935}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE), FrozenTrial(number=98, value=6.033698358075162, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 822061), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 824746), params={'x': 4.45635876005016}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE), FrozenTrial(number=99, value=1.8375623262943508, datetime_start=datetime.datetime(2020, 9, 7, 4, 28, 57, 825014), datetime_complete=datetime.datetime(2020, 9, 7, 4, 28, 57, 827909), params={'x': 0.6444328396223407}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

Total running time of the script: ( 0 minutes 0.629 seconds)

Gallery generated by Sphinx-Gallery