Lightweight, versatile, and platform agnostic architecture

Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.

Quadratic Function Example

Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).

First of all, import optuna.

import optuna

In optuna, conventionally functions to be optimized are named objective.

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.0034787348688905, (x - 2)^2: 1.2101596288034594e-05

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the dictionary of parameter name and parameter values:

Out:

{'x': 2.0034787348688905}

To get the best observed value of the objective function:

study.best_value

Out:

1.2101596288034594e-05

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=2, values=[1.2101596288034594e-05], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 780277), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 780425), params={'x': 2.0034787348688905}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, values=[124.39944888824962], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 779379), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 779596), params={'x': -9.153450089019524}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[33.69485994169083], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 779872), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 780027), params={'x': -3.8047273787569758}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[1.2101596288034594e-05], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 780277), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 780425), params={'x': 2.0034787348688905}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[8.665615400737144], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 780649), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 780796), params={'x': 4.94374173472082}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[20.218210443071833], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 781026), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 781177), params={'x': -2.4964664396692466}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[138.6327098974624], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 781396), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 781543), params={'x': -9.774239249202575}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[106.35005328364215], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 781761), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 781909), params={'x': -8.312616219158073}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[1.4147289284329667], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 782126), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 782276), params={'x': 3.189423780001462}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[46.679853240043336], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 782494), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 782647), params={'x': 8.83226560081232}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[57.60183801726061], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 782861), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 783026), params={'x': -5.5895874734573425}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[0.3016199045184491], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 783239), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 786035), params={'x': 1.4508006695939542}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[0.28661133471229494], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 786302), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 788914), params={'x': 1.4646390612752038}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[19.868291525769358], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 789198), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 791819), params={'x': 6.457386176423281}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[5.612876099016689], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 792082), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 794782), params={'x': -0.3691509236468429}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[3.0435640113607043], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 795047), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 797624), params={'x': 0.2554186716118092}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[4.807937248402586], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 797885), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 800438), params={'x': 4.192700902631863}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[63.38353047465644], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 800694), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 803505), params={'x': 9.961377423201116}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[24.780054016231713], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 803767), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 806350), params={'x': 6.977956811406836}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[0.4647278814307983], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 806608), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 809198), params={'x': 2.681709528634299}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[8.348106605543682], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 809458), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 812029), params={'x': -0.8893090187004371}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[20.150567962669513], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 812291), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 814890), params={'x': -2.4889384004093342}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[0.43170231506272483], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 815149), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 817744), params={'x': 1.3429594266236484}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[0.17360181306169928], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 818006), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 820573), params={'x': 1.5833444911420236}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[0.00020597306843650077], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 820832), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 823433), params={'x': 2.0143517618582703}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[12.485709851004676], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 823696), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 826301), params={'x': 5.533512395762137}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[3.082117424745997], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 826563), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 829158), params={'x': 3.7555960311945333}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[10.633488047498652], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 829419), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 832001), params={'x': -1.2609029497209285}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[0.16419478332947043], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 832262), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 834882), params={'x': 2.405209554834866}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[24.938853423057758], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 835143), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 837811), params={'x': 6.993881598822479}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[45.31010250198949], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 838073), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 840692), params={'x': -4.731277924880943}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[1.4925399057948714], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 840968), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 843620), params={'x': 3.22169550453248}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.004766978431445579], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 843883), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 846619), params={'x': 1.930956691624419}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[2.847347011156464], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 846882), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 849571), params={'x': 0.3125916288116668}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[0.0005881894911186297], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 849867), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 852576), params={'x': 1.975747381767763}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[24.818868227474642], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 852878), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 855610), params={'x': -2.981853894633467}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[8.427607897637868], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 855908), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 858650), params={'x': 4.903034257055515}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[1.4813684486062808], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 858952), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 861696), params={'x': 0.7828851949769566}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[13.4665202015292], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 861999), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 865035), params={'x': -1.6696757624522087}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[0.21873588113798956], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 865337), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 868024), params={'x': 2.467692079404804}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[6.503110704490816], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 868290), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 870925), params={'x': 4.550119743167135}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[42.669702593370936], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 871280), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 873995), params={'x': 8.53220503301687}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.16039121837654477], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 874264), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 876886), params={'x': 2.4004887244062494}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[0.0006505656340745286], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 877175), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 879772), params={'x': 2.0255061881525744}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[14.633919252491001], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 880041), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 882689), params={'x': 5.8254305970035585}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[2.3250289597202163], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 882993), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 885766), params={'x': 3.5248045644344774}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[6.005927419065544], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 886070), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 888998), params={'x': -0.45069937345761424}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[1.4587059316568054], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 889301), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 892230), params={'x': 0.7922310106411883}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[0.07753221313930982], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 892497), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 895222), params={'x': 1.7215539313631636}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[3.8117163143964854], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 895506), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 898846), params={'x': 3.9523617273437024}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[3.917011508648493], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 899174), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 903803), params={'x': 0.020855864610034747}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[16.94325477391443], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 904096), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 906863), params={'x': -2.116218504150919}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.01437470113669461], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 907137), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 909782), params={'x': 1.8801054582698002}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[0.5368947042052136], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 910060), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 912796), params={'x': 2.7327309903403934}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[0.020132143103982387], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 913095), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 915790), params={'x': 2.141887783490977}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[1.505748457531553], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 916068), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 918786), params={'x': 0.7729105747617442}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[1.6433748448344512], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 919059), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 921728), params={'x': 3.2819418258386186}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[0.48232157008699966], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 922037), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 924784), params={'x': 1.3055062490655516}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[7.251266114915906], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 925075), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 927782), params={'x': -0.6928175049408578}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[6.085251538187167], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 928056), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 930784), params={'x': 4.4668302613246755}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[0.018570551013779177], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 931063), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 933807), params={'x': 1.8637261910205076}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[10.584934878517247], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 934083), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 936799), params={'x': 5.2534496889482165}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.008406664447130103], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 937094), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 939794), params={'x': 1.9083121357696118}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[0.9877677172706661], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 940069), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 942813), params={'x': 1.0061349602331984}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[0.8191691274511671], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 943103), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 945838), params={'x': 2.905079624923226}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[2.8971582390327866], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 946113), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 948848), params={'x': 0.2978959376604535}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[3.3643464866351764], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 949148), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 951883), params={'x': 3.8342154962367907}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[0.0006397586555198865], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 952162), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 954924), params={'x': 2.025293450842459}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[0.9742772510465303], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 955201), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 957941), params={'x': 2.9870548368994148}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[88.3752728038104], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 958217), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 960978), params={'x': -7.400812348079841}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[4.98028329792492], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 961254), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 963999), params={'x': -0.2316548339572857}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[0.463789769801511], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 964276), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 967024), params={'x': 1.3189788771253046}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.00011541986713071305], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 967302), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 970081), params={'x': 2.010743363864764}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[0.0013926091434010197], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 970359), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 973147), params={'x': 2.0373176786979177}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[2.369796644515531], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 973430), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 976192), params={'x': 0.4605856163737032}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[9.63512354445382], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 976471), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 979228), params={'x': -1.1040495396262315}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[0.18148536986387626], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 979514), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 982289), params={'x': 2.426010997350862}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[1.9137454223777546], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 982570), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 985622), params={'x': 3.383381878722486}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[5.126098649984851], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 985906), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 988657), params={'x': 4.264088922720318}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[0.858769067292471], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 988934), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 991720), params={'x': 1.0733020625400793}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[0.031776602976538185], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 992002), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 995647), params={'x': 2.178259930933842}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[0.6722519983993677], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 995925), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 31, 998713), params={'x': 2.819909750155081}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.08495494264921075], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 31, 998994), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 1780), params={'x': 1.7085296882198622}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[0.0021626552122937176], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 2059), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 4891), params={'x': 2.0465043569173225}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[0.3184399858269398], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 5184), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 7950), params={'x': 1.4356951304242185}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[2.2910909582690335], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 8231), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 11018), params={'x': 3.5136350148794238}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[3.8888955190188352], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 11301), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 14099), params={'x': 0.027971724589417768}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[1.8095214548465952], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 14382), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 17177), params={'x': 0.6548154569552194}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[0.04437718894094629], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 17459), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 20224), params={'x': 2.21065893985527}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[8.374857283054215], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 20508), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 23338), params={'x': 4.893934567859857}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[0.5192107557101606], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 23624), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 26466), params={'x': 2.720562804833944}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[0.8193283148193454], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 26749), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 29576), params={'x': 1.094832438263862}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.019502264537288216], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 29862), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 32834), params={'x': 1.8603494914535281}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[0.010056913713096636], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 33152), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 35947), params={'x': 2.1002841648172663}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[1.4011984943555515], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 36237), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 39034), params={'x': 3.1837223045780423}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[0.25913477284066483], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 39325), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 42138), params={'x': 2.509052819303326}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[0.2016870894322535], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 42425), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 45257), params={'x': 1.5509041422677632}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[3.5809411499764017], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 45552), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 48360), params={'x': 3.892337483108233}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[1.9559174397901224], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 48648), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 51467), params={'x': 0.6014588172706095}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[4.580667185227306], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 51753), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 54594), params={'x': -0.1402493278184449}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[0.8728567443491442], datetime_start=datetime.datetime(2021, 2, 16, 12, 59, 32, 54883), datetime_complete=datetime.datetime(2021, 2, 16, 12, 59, 32, 57754), params={'x': 2.934268025969606}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.0034787348688905, (x - 2)^2: 1.2101596288034594e-05

Total running time of the script: ( 0 minutes 0.630 seconds)

Gallery generated by Sphinx-Gallery