Lightweight, versatile, and platform agnostic architecture

Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.

Quadratic Function Example

Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).

First of all, import optuna.

import optuna

In optuna, conventionally functions to be optimized are named objective.

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 1.9711403201212547, (x - 2)^2: 0.0008328811227036546

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the dictionary of parameter name and parameter values:

Out:

{'x': 1.9711403201212547}

To get the best observed value of the objective function:

study.best_value

Out:

0.0008328811227036546

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=84, values=[0.0008328811227036546], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 958648), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 962596), params={'x': 1.9711403201212547}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, values=[62.236137611765024], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 657372), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 657658), params={'x': 9.888988376957151}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[3.5588212263338814], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 658036), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 658263), params={'x': 0.11351617384779011}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[102.10905576730109], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 658635), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 658858), params={'x': -8.104902560999838}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[43.819700590744496], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 659201), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 659434), params={'x': -4.619645050208092}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[41.15646222665592], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 659738), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 659969), params={'x': 8.415330250786464}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[4.66598054561782], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 660311), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 660527), params={'x': -0.16008808746722636}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[6.884073245054106], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 660849), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 661069), params={'x': 4.623751749890623}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[15.148107691815724], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 661393), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 661622), params={'x': 5.892057000072805}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[2.3094220091087885], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 661950), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 662181), params={'x': 3.519678258418139}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[0.24725364649966886], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 662491), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 662722), params={'x': 1.502753937673038}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[15.510688965911077], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 663053), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 666724), params={'x': -1.938361203078138}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[0.9016829593172434], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 667061), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 670137), params={'x': 2.9495698812184616}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[0.13033710825506611], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 670441), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 673796), params={'x': 1.6389776900867952}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[35.40088020050116], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 674102), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 677500), params={'x': -3.9498638808380444}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[0.529933380215497], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 677826), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 681207), params={'x': 1.2720347671656997}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[22.905059503153105], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 681532), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 684974), params={'x': 6.785923056543336}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[23.35664247946882], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 685304), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 688779), params={'x': -2.832871038985918}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[75.00944036218094], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 689103), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 692522), params={'x': -6.660799060258871}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[0.05601221043984431], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 692846), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 696588), params={'x': 1.7633310108192366}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[10.822292164640897], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 696914), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 700306), params={'x': -1.289725241511956}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[0.010108511834002262], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 700611), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 704482), params={'x': 2.1005410952496653}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[0.8109641234815467], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 704807), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 708164), params={'x': 2.900535464866069}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[0.8662409388041011], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 708486), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 711891), params={'x': 1.0692793443765733}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[7.261474609552873], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 712229), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 715432), params={'x': 4.6947123426356425}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[0.0010574194241331075], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 715768), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 719174), params={'x': 1.9674820138364457}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[20.092142755940905], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 719506), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 723212), params={'x': 6.482425990012652}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[9.31810052137347], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 723580), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 727072), params={'x': -1.0525563911864873}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[3.5971877094478497], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 727403), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 730861), params={'x': 3.8966253476761956}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[0.11837233061504047], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 731209), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 734684), params={'x': 2.344052802074101}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[2.920761500317454], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 735033), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 738510), params={'x': 0.29097644828473657}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[51.04270282390579], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 738855), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 742398), params={'x': 9.144417598650417}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.09159092482088217], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 742730), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 746249), params={'x': 2.3026399260191592}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[8.6391645641606], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 746584), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 750672), params={'x': 4.939245577382162}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[4.180328811339027], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 751056), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 754631), params={'x': -0.04458524188624313}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[0.08768936354994061], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 754964), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 758532), params={'x': 2.296123898984767}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[30.873293939967546], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 758868), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 762403), params={'x': 7.556374172062888}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[2.282746043260728], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 762738), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 766282), params={'x': 0.4891240807860071}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[3.6788242490727563], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 766619), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 770159), params={'x': 3.9180261335739814}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[6.828182902990965], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 770496), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 774038), params={'x': -0.6130791995251434}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[12.49802286713212], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 774375), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 777871), params={'x': 5.535254286063751}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[23.8880782715364], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 778206), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 781631), params={'x': -2.8875431733680266}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.04430965683793547], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 781968), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 785537), params={'x': 2.2104985910592645}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[0.14477450743038295], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 785890), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 789480), params={'x': 2.380492453841575}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[2.532454504412555], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 789818), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 793327), params={'x': 3.5913687518650588}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[1.545051356436472], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 793666), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 797558), params={'x': 0.7569990521176293}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[0.0666109048697983], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 797897), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 801414), params={'x': 1.741909115097417}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[0.24874416914100658], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 801754), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 805345), params={'x': 1.5012574119437898}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[18.749728567809665], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 805683), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 809273), params={'x': -2.330095676519131}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[1.5035469352552928], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 809615), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 813218), params={'x': 3.2261920466449343}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[5.991616554790701], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 813556), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 817157), params={'x': -0.4477778810159023}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[5.505386853245152], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 817497), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 822117), params={'x': 4.346356079806548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.053323628920229404], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 822603), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 826406), params={'x': 2.2309190960493077}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[0.0972398666139348], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 826753), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 832841), params={'x': 1.6881669250802045}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[0.6726358366049145], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 833341), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 838424), params={'x': 2.820143790200788}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[12.133118804469719], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 838797), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 842450), params={'x': 5.4832626665914415}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[0.07025360069615909], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 842795), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 846529), params={'x': 1.7349460419156901}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[1.364160037067698], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 846910), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 850448), params={'x': 0.8320273817132107}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[12.961451609393412], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 850799), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 854420), params={'x': -1.6002016067705727}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[1.082630759475878], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 854766), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 858368), params={'x': 3.0404954394306003}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[141.0713516018902], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 858723), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 862322), params={'x': -9.877346151472146}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[3.9510214757863773], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 862656), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 866250), params={'x': 3.9877176549465916}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.03876603681293981], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 866596), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 870144), params={'x': 1.8031090738176596}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[0.0016518419855895848], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 870469), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 874045), params={'x': 1.9593571410258876}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[2.969759633400968], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 874395), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 878008), params={'x': 0.2767009448731865}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[0.041369124068006934], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 878358), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 881972), params={'x': 2.20339401187844}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[0.8185379605074382], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 882324), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 885946), params={'x': 1.0952691226074804}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[0.39736355388601763], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 886297), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 889757), params={'x': 2.630367792551315}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[9.626560622107796], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 890111), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 893777), params={'x': 5.102669918329663}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[7.193464103888897], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 894127), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 898186), params={'x': -0.6820634041515308}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[1.494883473584289], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 898540), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 902119), params={'x': 3.222654273940221}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[18.572295032871807], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 902469), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 905995), params={'x': 6.309558565894169}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.004987693229913129], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 906347), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 909943), params={'x': 1.9293763975011673}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.11088992296050558], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 910292), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 913941), params={'x': 2.3330013858237013}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[0.7523458550098076], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 914296), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 917930), params={'x': 1.1326212736008523}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[4.128565427003404], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 918287), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 921946), params={'x': -0.031887159023207934}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[5.843453896455969], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 922301), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 925954), params={'x': 4.417323705351844}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[2.5679221565760333], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 926309), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 930017), params={'x': 3.602473761587388}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[0.0036793778732600664], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 930374), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 934029), params={'x': 2.060657875607872}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[2.070978779178735], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 934374), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 938111), params={'x': 0.5609104339275004}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[0.04896347770252587], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 938474), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 942084), params={'x': 1.7787230746270053}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[0.44333130508058255], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 942447), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 946032), params={'x': 1.3341687112484255}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.00539640437548078], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 946415), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 950128), params={'x': 2.0734602230835217}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[0.004493839820215944], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 950484), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 954213), params={'x': 2.0670361083313757}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[1.015648083385248], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 954569), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 958292), params={'x': 3.007793671038496}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[0.0008328811227036546], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 958648), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 962596), params={'x': 1.9711403201212547}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[1.8241655417207785], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 963079), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 966947), params={'x': 0.6493832735669314}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[2.467875693859409], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 967328), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 971349), params={'x': 3.570947387361973}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[0.007149796217025023], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 971683), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 975460), params={'x': 1.9154435323761392}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[4.50260601854136], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 976750), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 980485), params={'x': -0.12193449911663357}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[0.6248944555939134], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 980845), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 984559), params={'x': 2.790502660080226}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[4.55701554045824], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 984956), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 988697), params={'x': 4.134716735414383}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.013659809746195215], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 989060), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 992756), params={'x': 1.883124811246376}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[1.1115823065731307], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 993119), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 12, 996830), params={'x': 0.9456839626691005}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[0.6524191955651099], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 12, 997191), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 835), params={'x': 1.1922752971679584}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[0.003991262446918265], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 1194), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 4866), params={'x': 2.063176439017392}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[0.4649000768947512], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 5230), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 8992), params={'x': 2.6818358137372598}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[1.6925433669085481], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 9351), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 13086), params={'x': 3.3009778502759177}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[0.01347450783780069], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 13448), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 17201), params={'x': 2.116079747750418}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[8.854838840886162], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 17562), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 21313), params={'x': -0.9757081242766672}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[0.3735066125995011], datetime_start=datetime.datetime(2021, 2, 1, 6, 20, 13, 21675), datetime_complete=datetime.datetime(2021, 2, 1, 6, 20, 13, 25424), params={'x': 1.3888481264043273}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.0024482156595917, (x - 2)^2: 5.993759915870247e-06

Total running time of the script: ( 0 minutes 0.834 seconds)

Gallery generated by Sphinx-Gallery