Lightweight, versatile, and platform agnostic architecture

Optuna is entirely written in Python and has few dependencies. This means that we can quickly move to the real example once you get interested in Optuna.

Quadratic Function Example

Usually, Optuna is used to optimize hyperparameters, but as an example, let’s optimize a simple quadratic function: \((x - 2)^2\).

First of all, import optuna.

import optuna

In optuna, conventionally functions to be optimized are named objective.

def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return (x - 2) ** 2

This function returns the value of \((x - 2)^2\). Our goal is to find the value of x that minimizes the output of the objective function. This is the “optimization.” During the optimization, Optuna repeatedly calls and evaluates the objective function with different values of x.

A Trial object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (for example, suggest_float()) are called inside the objective function to obtain parameters for a trial. suggest_float() selects parameters uniformly within the range provided. In our example, from \(-10\) to \(10\).

To start the optimization, we create a study object and pass the objective function to method optimize() as follows.

study = optuna.create_study()
study.optimize(objective, n_trials=100)

You can get the best parameter as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 1.9834632616820875, (x - 2)^2: 0.00027346371419511455

We can see that the x value found by Optuna is close to the optimal value of 2.

Note

When used to search for hyperparameters in machine learning, usually the objective function would return the loss or accuracy of the model.

Study Object

Let us clarify the terminology in Optuna as follows:

  • Trial: A single call of the objective function

  • Study: An optimization session, which is a set of trials

  • Parameter: A variable whose value is to be optimized, such as x in the above example

In Optuna, we use the study object to manage optimization. Method create_study() returns a study object. A study object has useful properties for analyzing the optimization outcome.

To get the dictionary of parameter name and parameter values:

Out:

{'x': 1.9834632616820875}

To get the best observed value of the objective function:

study.best_value

Out:

0.00027346371419511455

To get the best trial:

study.best_trial

Out:

FrozenTrial(number=81, values=[0.00027346371419511455], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 62622), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 66122), params={'x': 1.9834632616820875}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None)

To get all trials:

study.trials

Out:

[FrozenTrial(number=0, values=[1.79890940506639], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 776597), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 776840), params={'x': 0.658765715817557}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=0, state=TrialState.COMPLETE, value=None), FrozenTrial(number=1, values=[13.734056619709436], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 777382), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 777526), params={'x': 5.705948815041761}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=1, state=TrialState.COMPLETE, value=None), FrozenTrial(number=2, values=[23.216922910712146], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 777974), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 778109), params={'x': 6.818394225331936}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=2, state=TrialState.COMPLETE, value=None), FrozenTrial(number=3, values=[40.2887923833743], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 778550), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 778690), params={'x': -4.347345302043548}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=3, state=TrialState.COMPLETE, value=None), FrozenTrial(number=4, values=[44.892473765017925], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 779128), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 779264), params={'x': -4.700184606786437}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=4, state=TrialState.COMPLETE, value=None), FrozenTrial(number=5, values=[9.48361592599732], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 779706), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 779855), params={'x': 5.0795480067693894}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=5, state=TrialState.COMPLETE, value=None), FrozenTrial(number=6, values=[22.537794600854465], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 780290), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 780429), params={'x': 6.747398719388805}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=6, state=TrialState.COMPLETE, value=None), FrozenTrial(number=7, values=[2.7505130573000236], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 780864), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 781005), params={'x': 3.6584670805596424}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=7, state=TrialState.COMPLETE, value=None), FrozenTrial(number=8, values=[14.282900988301463], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 781438), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 781582), params={'x': -1.7792725475018951}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=8, state=TrialState.COMPLETE, value=None), FrozenTrial(number=9, values=[5.914809566339953], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 782009), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 782156), params={'x': -0.43203815067526286}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=9, state=TrialState.COMPLETE, value=None), FrozenTrial(number=10, values=[104.78086596773029], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 782582), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 786136), params={'x': -8.236252535363237}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=10, state=TrialState.COMPLETE, value=None), FrozenTrial(number=11, values=[0.3206618758435928], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 786604), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 789909), params={'x': 2.566270143874452}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=11, state=TrialState.COMPLETE, value=None), FrozenTrial(number=12, values=[0.033404958654207256], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 790375), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 793678), params={'x': 2.1827702345958095}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=12, state=TrialState.COMPLETE, value=None), FrozenTrial(number=13, values=[0.09026415272172954], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 794144), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 797377), params={'x': 2.30043993196932}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=13, state=TrialState.COMPLETE, value=None), FrozenTrial(number=14, values=[59.93493374434304], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 797842), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 801851), params={'x': 9.741765544392509}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=14, state=TrialState.COMPLETE, value=None), FrozenTrial(number=15, values=[0.004638795508322752], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 802319), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 805525), params={'x': 2.0681087036165184}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=15, state=TrialState.COMPLETE, value=None), FrozenTrial(number=16, values=[47.443346554378486], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 805986), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 809239), params={'x': 8.887913076859963}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=16, state=TrialState.COMPLETE, value=None), FrozenTrial(number=17, values=[14.359603343755136], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 809710), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 812947), params={'x': -1.7894067271480818}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=17, state=TrialState.COMPLETE, value=None), FrozenTrial(number=18, values=[1.6369683072087968], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 813411), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 816649), params={'x': 0.7205593772242509}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=18, state=TrialState.COMPLETE, value=None), FrozenTrial(number=19, values=[46.55670944571525], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 817118), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 820382), params={'x': -4.823247719797021}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=19, state=TrialState.COMPLETE, value=None), FrozenTrial(number=20, values=[3.2689584099472224], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 820846), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 824104), params={'x': 3.8080261087570673}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=20, state=TrialState.COMPLETE, value=None), FrozenTrial(number=21, values=[0.06523446788205621], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 824564), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 827826), params={'x': 1.744589608899606}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=21, state=TrialState.COMPLETE, value=None), FrozenTrial(number=22, values=[0.0002735135891566531], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 828289), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 831530), params={'x': 2.016538246253961}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=22, state=TrialState.COMPLETE, value=None), FrozenTrial(number=23, values=[18.04404722686849], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 832005), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 835257), params={'x': -2.2478285307752817}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=23, state=TrialState.COMPLETE, value=None), FrozenTrial(number=24, values=[4.762940096016476], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 835733), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 839507), params={'x': 4.182416114313784}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=24, state=TrialState.COMPLETE, value=None), FrozenTrial(number=25, values=[2.587070374249796], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 839988), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 843764), params={'x': 0.3915627540218438}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=25, state=TrialState.COMPLETE, value=None), FrozenTrial(number=26, values=[30.503583832168307], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 844228), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 847441), params={'x': 7.5230049639818635}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=26, state=TrialState.COMPLETE, value=None), FrozenTrial(number=27, values=[0.5114359572105788], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 847942), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 851191), params={'x': 2.7151475073092115}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=27, state=TrialState.COMPLETE, value=None), FrozenTrial(number=28, values=[26.513796366438548], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 851656), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 854982), params={'x': -3.1491549176965483}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=28, state=TrialState.COMPLETE, value=None), FrozenTrial(number=29, values=[3.962044826112616], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 855448), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 858764), params={'x': 0.009511410202857729}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=29, state=TrialState.COMPLETE, value=None), FrozenTrial(number=30, values=[0.7984985565017142], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 859232), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 862587), params={'x': 1.1064125356174057}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=30, state=TrialState.COMPLETE, value=None), FrozenTrial(number=31, values=[0.05815036543372646], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 863054), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 866415), params={'x': 1.7588561312541278}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=31, state=TrialState.COMPLETE, value=None), FrozenTrial(number=32, values=[7.12023205461177], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 866883), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 870255), params={'x': -0.668376295542248}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=32, state=TrialState.COMPLETE, value=None), FrozenTrial(number=33, values=[9.973343891250408], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 870724), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 874097), params={'x': 5.158060146870292}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=33, state=TrialState.COMPLETE, value=None), FrozenTrial(number=34, values=[1.5551119720407802], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 874564), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 877946), params={'x': 3.2470412872237953}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=34, state=TrialState.COMPLETE, value=None), FrozenTrial(number=35, values=[0.20290871884190126], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 878442), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 881832), params={'x': 1.5495460968734966}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=35, state=TrialState.COMPLETE, value=None), FrozenTrial(number=36, values=[16.0607859423861], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 882302), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 885711), params={'x': 6.007591039812583}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=36, state=TrialState.COMPLETE, value=None), FrozenTrial(number=37, values=[7.3981769027451305], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 886179), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 889619), params={'x': 4.719958989166037}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=37, state=TrialState.COMPLETE, value=None), FrozenTrial(number=38, values=[7.526450575215767], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 890120), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 893540), params={'x': -0.7434377294219323}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=38, state=TrialState.COMPLETE, value=None), FrozenTrial(number=39, values=[0.0004017704448661392], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 894063), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 897486), params={'x': 2.0200442122535693}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=39, state=TrialState.COMPLETE, value=None), FrozenTrial(number=40, values=[1.007389641119402], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 897952), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 901387), params={'x': 3.0036880198146245}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=40, state=TrialState.COMPLETE, value=None), FrozenTrial(number=41, values=[0.04617613791314271], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 901853), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 905276), params={'x': 1.7851136628048616}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=41, state=TrialState.COMPLETE, value=None), FrozenTrial(number=42, values=[0.9607923824389241], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 905745), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 909208), params={'x': 1.0197998253219274}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=42, state=TrialState.COMPLETE, value=None), FrozenTrial(number=43, values=[4.795393511562654], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 910283), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 913700), params={'x': 4.1898386953295566}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=43, state=TrialState.COMPLETE, value=None), FrozenTrial(number=44, values=[0.07492840743193055], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 914169), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 917646), params={'x': 2.273730537996641}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=44, state=TrialState.COMPLETE, value=None), FrozenTrial(number=45, values=[3.604056031694071], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 918116), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 921577), params={'x': 0.10156484659231224}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=45, state=TrialState.COMPLETE, value=None), FrozenTrial(number=46, values=[15.831746646753341], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 922048), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 925482), params={'x': 5.978912746813298}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=46, state=TrialState.COMPLETE, value=None), FrozenTrial(number=47, values=[8.016182676176834], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 925952), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 929484), params={'x': -0.8312863995323458}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=47, state=TrialState.COMPLETE, value=None), FrozenTrial(number=48, values=[1.7364879162980764], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 929955), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 933392), params={'x': 3.3177586714941687}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=48, state=TrialState.COMPLETE, value=None), FrozenTrial(number=49, values=[26.732623910046783], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 933860), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 937292), params={'x': 7.170360133496194}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=49, state=TrialState.COMPLETE, value=None), FrozenTrial(number=50, values=[0.035387612125370484], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 937767), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 941214), params={'x': 2.188115953936317}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=50, state=TrialState.COMPLETE, value=None), FrozenTrial(number=51, values=[0.03084035188376526], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 941686), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 945854), params={'x': 1.8243857867831728}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=51, state=TrialState.COMPLETE, value=None), FrozenTrial(number=52, values=[0.16967718186533776], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 946333), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 949786), params={'x': 2.411918902049102}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=52, state=TrialState.COMPLETE, value=None), FrozenTrial(number=53, values=[1.2479472271546008], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 950260), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 953692), params={'x': 0.8828844163853944}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=53, state=TrialState.COMPLETE, value=None), FrozenTrial(number=54, values=[6.668444855224021], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 954169), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 957600), params={'x': 4.582333219246506}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=54, state=TrialState.COMPLETE, value=None), FrozenTrial(number=55, values=[2.597558228403569], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 958073), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 962056), params={'x': 3.6116942105758056}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=55, state=TrialState.COMPLETE, value=None), FrozenTrial(number=56, values=[0.06257901664788754], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 962530), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 965987), params={'x': 2.2501579833782794}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=56, state=TrialState.COMPLETE, value=None), FrozenTrial(number=57, values=[11.302437039904996], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 966461), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 969907), params={'x': -1.361909731076222}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=57, state=TrialState.COMPLETE, value=None), FrozenTrial(number=58, values=[11.163127092231218], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 970380), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 973816), params={'x': 5.341126620203314}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=58, state=TrialState.COMPLETE, value=None), FrozenTrial(number=59, values=[2.3853391758152176], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 974291), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 977748), params={'x': 0.45554567053110384}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=59, state=TrialState.COMPLETE, value=None), FrozenTrial(number=60, values=[28.34370127590917], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 978226), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 981690), params={'x': -3.323880283769459}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=60, state=TrialState.COMPLETE, value=None), FrozenTrial(number=61, values=[0.1970936276731797], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 982167), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 985613), params={'x': 1.5560477191485782}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=61, state=TrialState.COMPLETE, value=None), FrozenTrial(number=62, values=[0.10601027311878733], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 986090), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 989568), params={'x': 1.6744078116434804}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=62, state=TrialState.COMPLETE, value=None), FrozenTrial(number=63, values=[0.4513956344221182], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 990046), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 994079), params={'x': 2.671859832422}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=63, state=TrialState.COMPLETE, value=None), FrozenTrial(number=64, values=[3.4203716159692292], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 994554), datetime_complete=datetime.datetime(2021, 1, 12, 7, 41, 59, 998030), params={'x': 3.849424671612561}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=64, state=TrialState.COMPLETE, value=None), FrozenTrial(number=65, values=[0.6335657977454033], datetime_start=datetime.datetime(2021, 1, 12, 7, 41, 59, 998507), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 1988), params={'x': 1.2040315347041672}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=65, state=TrialState.COMPLETE, value=None), FrozenTrial(number=66, values=[4.644441749463895], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 2473), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 5947), params={'x': -0.1550966914419165}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=66, state=TrialState.COMPLETE, value=None), FrozenTrial(number=67, values=[137.21131502271965], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 6430), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 9900), params={'x': -9.713723362907272}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=67, state=TrialState.COMPLETE, value=None), FrozenTrial(number=68, values=[0.00029035120258541887], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 10380), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 13861), params={'x': 2.0170396949088127}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=68, state=TrialState.COMPLETE, value=None), FrozenTrial(number=69, values=[0.888984396589363], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 14343), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 17836), params={'x': 2.9428596908285787}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=69, state=TrialState.COMPLETE, value=None), FrozenTrial(number=70, values=[0.01709694991352161], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 18314), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 21797), params={'x': 2.1307553054889996}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=70, state=TrialState.COMPLETE, value=None), FrozenTrial(number=71, values=[0.027132587284457447], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 22275), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 25757), params={'x': 2.1647197234227202}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=71, state=TrialState.COMPLETE, value=None), FrozenTrial(number=72, values=[2.3571560242436833], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 26235), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 29717), params={'x': 0.4646967647257161}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=72, state=TrialState.COMPLETE, value=None), FrozenTrial(number=73, values=[1.863513807200368], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 30192), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 33665), params={'x': 3.365105786084129}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=73, state=TrialState.COMPLETE, value=None), FrozenTrial(number=74, values=[4.956854615568224], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 34143), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 38196), params={'x': 4.226399473492622}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=74, state=TrialState.COMPLETE, value=None), FrozenTrial(number=75, values=[0.3917511602905471], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 38678), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 42180), params={'x': 1.3740997201705794}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=75, state=TrialState.COMPLETE, value=None), FrozenTrial(number=76, values=[0.7206756341896277], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 42663), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 46159), params={'x': 2.8489261653345523}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=76, state=TrialState.COMPLETE, value=None), FrozenTrial(number=77, values=[0.008796812872613219], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 46639), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 50151), params={'x': 2.0937913262120396}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=77, state=TrialState.COMPLETE, value=None), FrozenTrial(number=78, values=[0.014606887462145885], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 50631), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 54144), params={'x': 2.1208589568966483}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=78, state=TrialState.COMPLETE, value=None), FrozenTrial(number=79, values=[2.3185575165384784], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 54625), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 58136), params={'x': 0.47731897084830055}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=79, state=TrialState.COMPLETE, value=None), FrozenTrial(number=80, values=[1.4685395640772152], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 58619), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 62142), params={'x': 0.7881668579886028}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=80, state=TrialState.COMPLETE, value=None), FrozenTrial(number=81, values=[0.00027346371419511455], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 62622), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 66122), params={'x': 1.9834632616820875}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=81, state=TrialState.COMPLETE, value=None), FrozenTrial(number=82, values=[0.0006258448955001921], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 66603), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 70153), params={'x': 1.9749831077969267}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=82, state=TrialState.COMPLETE, value=None), FrozenTrial(number=83, values=[5.535165744726199], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 70632), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 74173), params={'x': -0.3526932959325997}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=83, state=TrialState.COMPLETE, value=None), FrozenTrial(number=84, values=[3.002516116261062], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 74660), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 78730), params={'x': 3.7327769955366623}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=84, state=TrialState.COMPLETE, value=None), FrozenTrial(number=85, values=[1.0755317891110918], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 79220), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 82764), params={'x': 3.037078487440122}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=85, state=TrialState.COMPLETE, value=None), FrozenTrial(number=86, values=[0.4837297497946812], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 83250), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 86776), params={'x': 1.3044931705621567}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=86, state=TrialState.COMPLETE, value=None), FrozenTrial(number=87, values=[0.3780655625288568], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 87260), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 90807), params={'x': 2.6148703623763767}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=87, state=TrialState.COMPLETE, value=None), FrozenTrial(number=88, values=[10.377333541824195], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 91295), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 94860), params={'x': -1.2213868972577937}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=88, state=TrialState.COMPLETE, value=None), FrozenTrial(number=89, values=[0.00048673879489408265], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 95347), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 98908), params={'x': 2.0220621575303523}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=89, state=TrialState.COMPLETE, value=None), FrozenTrial(number=90, values=[3.7632046387609277], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 99394), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 102953), params={'x': 0.06010189990274806}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=90, state=TrialState.COMPLETE, value=None), FrozenTrial(number=91, values=[0.005853196991702883], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 103439), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 106991), params={'x': 2.0765061892378838}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=91, state=TrialState.COMPLETE, value=None), FrozenTrial(number=92, values=[7.102580606645376], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 107477), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 111030), params={'x': 4.665066717109607}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=92, state=TrialState.COMPLETE, value=None), FrozenTrial(number=93, values=[1.1435504968380912], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 111516), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 115093), params={'x': 0.9306307948897672}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=93, state=TrialState.COMPLETE, value=None), FrozenTrial(number=94, values=[0.016548465209923938], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 115578), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 119155), params={'x': 1.8713591619666448}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=94, state=TrialState.COMPLETE, value=None), FrozenTrial(number=95, values=[0.31218551739769423], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 119637), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 123819), params={'x': 2.558735641782135}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=95, state=TrialState.COMPLETE, value=None), FrozenTrial(number=96, values=[1.3412445627123], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 124306), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 128302), params={'x': 3.1581211347317257}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=96, state=TrialState.COMPLETE, value=None), FrozenTrial(number=97, values=[0.4683080593402442], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 128791), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 132391), params={'x': 1.3156696270511996}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=97, state=TrialState.COMPLETE, value=None), FrozenTrial(number=98, values=[4.1060588367689625], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 132882), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 136995), params={'x': 4.026341243909565}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=98, state=TrialState.COMPLETE, value=None), FrozenTrial(number=99, values=[0.0005475196793137636], datetime_start=datetime.datetime(2021, 1, 12, 7, 42, 0, 137484), datetime_complete=datetime.datetime(2021, 1, 12, 7, 42, 0, 141086), params={'x': 2.023399138431014}, distributions={'x': UniformDistribution(high=10, low=-10)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=99, state=TrialState.COMPLETE, value=None)]

To get the number of trials:

len(study.trials)

Out:

100

By executing optimize() again, we can continue the optimization.

study.optimize(objective, n_trials=100)

To get the updated number of trials:

len(study.trials)

Out:

200

As the objective function is so easy that the last 100 trials don’t improve the result. However, we can check the result again:

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))

Out:

Found x: 2.0029789268074194, (x - 2)^2: 8.874004923961951e-06

Total running time of the script: ( 0 minutes 0.829 seconds)

Gallery generated by Sphinx-Gallery