Lightweight, versatile, and platform agnostic architecture

Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.

Quadratic Function Example

Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).

First of all, import optuna.

import optuna

In optuna, conventionally functions to be optimized are named objective.

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.0043483688860726, (x - 2)^2: 1.8908311969364224e-05

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the dictionary of parameter name and parameter values:

Out:

{'x': 2.0043483688860726}

To get the best observed value of the objective function:

study.best_value

Out:

1.8908311969364224e-05

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=70, values=[1.8908311969364224e-05], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 689925), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 693466), params={'x': 2.0043483688860726}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, values=[8.949756244555367], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 442560), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 442807), params={'x': -0.9916143208233521}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[4.359035338215807], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 443185), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 443453), params={'x': 4.087830294400339}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[20.550075077384953], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 443795), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 444049), params={'x': 6.533219063467477}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[133.40373490090693], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 444376), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 444628), params={'x': -9.550053458790003}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[51.709446680277985], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 444939), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 445190), params={'x': -5.190928081984827}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[79.60314941351464], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 445495), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 445736), params={'x': -6.922059706901464}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[0.522143261820939], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 446040), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 446296), params={'x': 2.7225948116482286}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[15.091559993165587], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 446601), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 446853), params={'x': -1.8847857074960501}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[100.46916119500611], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 447154), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 447406), params={'x': -8.023430610075879}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[1.9423438821540617], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 447703), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 447961), params={'x': 3.3936799783860216}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[56.29448638040105], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 448275), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 451873), params={'x': 9.502965172543522}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[1.906514986062257], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 452213), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 455574), params={'x': 3.380766086657062}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[0.12302730269858943], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 455890), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 459306), params={'x': 2.350752480673465}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[0.689733044884457], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 459626), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 463845), params={'x': 1.1694983173500146}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[34.36013591835597], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 464184), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 467520), params={'x': 7.861751949575824}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[1.3717246092425204], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 467838), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 471175), params={'x': 0.8287935240776201}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[27.699182675244234], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 471496), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 475909), params={'x': -3.263001299187017}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[12.98922820445288], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 476247), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 479603), params={'x': 5.604057186623553}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[0.002799640905870712], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 479921), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 483251), params={'x': 2.0529116329919113}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[28.180508828397304], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 483572), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 486964), params={'x': -3.308531701741764}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[0.6602463458960627], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 487285), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 490695), params={'x': 1.1874445582632145}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[0.5154006059022703], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 491018), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 494410), params={'x': 2.7179140658200467}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[16.072884233148127], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 494734), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 498133), params={'x': 6.009100177489723}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[4.897000417015468], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 498456), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 501851), params={'x': -0.21291672166294306}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[0.05948590295825489], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 502177), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 505568), params={'x': 1.7561026794770904}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[7.865721431527149], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 505892), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 509554), params={'x': 4.804589351674706}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[0.24162084952308457], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 509879), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 513270), params={'x': 1.5084505624831979}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[18.994062789702827], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 513614), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 517033), params={'x': -2.3582178455996012}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[29.183430521301453], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 517361), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 520824), params={'x': 7.402169057082669}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[7.874695351173404], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 521151), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 525015), params={'x': -0.8061887590063153}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[0.051216353512779283], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 525416), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 528905), params={'x': 2.2263103035939356}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.14751015210023227], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 529235), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 532782), params={'x': 1.6159294959252504}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[3.7255383658384877], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 533141), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 536665), params={'x': 0.06983462733410128}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[5.6241825245088055], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 537000), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 540426), params={'x': 4.371535899898799}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[0.06526784904234827], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 540754), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 544225), params={'x': 2.2554757308284845}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[6.2202722142105955], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 544561), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 548003), params={'x': 4.494047356048116}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[12.662462939056566], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 548457), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 553756), params={'x': -1.5584354622581769}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[4.6813215985901255], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 554086), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 557848), params={'x': -0.16363619829908704}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[2.32263962476263], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 558177), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 561468), params={'x': 3.524020874122999}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[0.048600902025715036], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 561797), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 565105), params={'x': 2.2204561226768607}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[25.84862786101739], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 565431), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 568958), params={'x': 7.0841545866562114}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.0063968048360297765], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 569290), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 572834), params={'x': 2.0799800277321143}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[1.7908393328546302], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 573180), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 576680), params={'x': 0.6617775473208387}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[10.03220343177631], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 577017), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 580497), params={'x': 5.167365377056507}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[2.794473757670328], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 580805), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 584256), params={'x': 3.6716679567636414}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[0.02880402866384944], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 584585), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 588104), params={'x': 2.1697174966344055}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[0.7078608496657035], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 588435), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 591918), params={'x': 2.8413446675802394}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[2.577355436784433], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 592270), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 595770), params={'x': 0.39458558721293624}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[43.5075490741569], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 596114), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 599643), params={'x': 8.596025248144286}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[8.346551117305683], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 599982), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 603456), params={'x': -0.8890398261889163}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[0.099179952489964], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 603794), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 607314), params={'x': 2.3149284878983862}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.060342205845553006], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 607649), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 611205), params={'x': 1.7543534941311947}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[1.4779818601467647], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 611549), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 615058), params={'x': 3.2157227727351185}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[4.191529286233388], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 615395), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 619243), params={'x': 4.047322467574023}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[0.01995029420118302], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 619579), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 623088), params={'x': 1.8587544896246857}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[1.7380158757418498], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 623426), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 626953), params={'x': 0.6816616990537484}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[0.1181654047433454], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 627420), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 631182), params={'x': 2.3437519523484127}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[0.8772180711655616], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 631522), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 635036), params={'x': 2.936599205191613}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[15.600244751713497], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 635376), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 638911), params={'x': 5.949714515216701}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[0.5627988092882192], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 639251), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 642874), params={'x': 1.249800820256234}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[22.239752560286664], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 643215), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 646737), params={'x': -2.715904214494466}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.20562357684833799], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 647083), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 650616), params={'x': 1.5465426405401077}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[0.00033884589146881993], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 650965), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 656346), params={'x': 1.9815922328494513}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[3.1864624171146794], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 656864), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 662875), params={'x': 3.785066502154662}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[5.77340856489054], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 663368), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 669947), params={'x': -0.4027918272065394}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[0.005879155301895235], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 670472), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 673953), params={'x': 1.9233243500066988}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[13.189428997038805], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 674302), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 677873), params={'x': -1.6317253471371984}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[1.3022224101861037], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 678214), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 681768), params={'x': 0.858850399734503}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[0.5739542288700388], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 682111), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 685666), params={'x': 2.7575976695252162}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[8.970486890269846], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 686009), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 689582), params={'x': 4.995077109236062}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[1.8908311969364224e-05], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 689925), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 693466), params={'x': 2.0043483688860726}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.009306903467171756], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 693813), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 697400), params={'x': 2.096472293779985}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.020183217563886274], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 697745), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 701308), params={'x': 1.8579323486366925}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[2.637930032193232], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 701653), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 705235), params={'x': 0.3758294325431113}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[0.7397037879703833], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 705582), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 709157), params={'x': 1.139939660273546}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[1.2074328878319471], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 709504), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 713373), params={'x': 3.098832511273646}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[0.05961605851586815], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 713719), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 717310), params={'x': 1.7558360007784355}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[5.175572552689062], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 717660), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 721233), params={'x': 4.274988473089273}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[7.134859191891114], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 721578), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 725155), params={'x': -0.6711157204230431}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[0.9155522574995208], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 725503), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 729156), params={'x': 1.0431550504394556}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[1.7803464396775783], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 729505), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 733097), params={'x': 3.3342962338542286}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.0352880024388402], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 733448), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 737047), params={'x': 2.187851011279791}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[3.31102935124998], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 737396), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 740985), params={'x': 0.18037659081611612}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[0.026238524767850577], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 741334), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 744933), params={'x': 1.8380168997461446}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[0.037284567184683946], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 745289), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 748931), params={'x': 1.806907879019666}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[0.3845754359766063], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 749281), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 752880), params={'x': 2.6201414644874235}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[3.191728322081699], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 753238), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 757366), params={'x': 3.7865408817269475}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[0.5114275096701366], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 757898), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 764407), params={'x': 1.2848583988676532}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[4.006460635495804], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 764990), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 770496), params={'x': -0.0016145072155639983}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[11.064209066050044], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 770855), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 774489), params={'x': -1.3262905865317967}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[0.0032209948820770802], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 774847), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 778500), params={'x': 1.9432461905941365}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[134.69753300952019], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 778855), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 782465), params={'x': -9.605926632954397}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[0.008691263755270134], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 782824), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 786457), params={'x': 1.906773052419002}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[0.43277541572472866], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 786814), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 790565), params={'x': 2.65785668327131}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[1.8587868198281905], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 790925), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 794562), params={'x': 0.6366266762811477}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[0.0006347026123511486], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 794919), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 798575), params={'x': 1.9748066950887513}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[1.0410744304344135], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 798945), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 802588), params={'x': 3.0203305495938135}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[1.9848857724338609], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 802943), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 806885), params={'x': 3.408859741930992}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[0.3345258409147058], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 807253), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 810916), params={'x': 1.421617910966543}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[0.27732225645138603], datetime_start=datetime.datetime(2021, 1, 12, 6, 34, 29, 811274), datetime_complete=datetime.datetime(2021, 1, 12, 6, 34, 29, 814909), params={'x': 2.52661395390873}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.002927317314964, (x - 2)^2: 8.56918666248925e-06

Total running time of the script: ( 0 minutes 0.842 seconds)

Gallery generated by Sphinx-Gallery