# First Optimization¶

## Quadratic Function Example¶

Let us try very simple optimization in IPython shell.

```
In [1]: import optuna
```

Here, we use a very simple quadratic function as an example of objective function.

```
In [2]: def objective(trial):
...: x = trial.suggest_uniform('x', -10, 10)
...: return (x - 2) ** 2
...:
```

Our goal is to find out `x`

that minimizes the output of `objective`

function, which we refer to as “optimization.” During the optimization, Optuna repeatedly invokes and evaluates the objective function with different values of `x`

.

A `Trial`

object corresponds to a single execution of the objective function and is internally instantiated upon each invocation of the function.

The suggest APIs (e.g., `suggest_uniform()`

) are called inside the objective function to obtain parameters for a trial.

To start the optimization, we create a study object and pass the objective function to method `optimize()`

as follows.

```
In [3]: study = optuna.create_study()
In [4]: study.optimize(objective, n_trials=100)
[I 2018-05-09 10:03:22,469] Finished a trial resulted in value: 52.9345515866657. Current best value is 52.9345515866657 with parameters: {'x': -5.275613485244093}.
[I 2018-05-09 10:03:22,474] Finished a trial resulted in value: 32.82718929591965. Current best value is 32.82718929591965 with parameters: {'x': -3.7295016620924066}.
[I 2018-05-09 10:03:22,475] Finished a trial resulted in value: 46.89428737068025. Current best value is 32.82718929591965 with parameters: {'x': -3.7295016620924066}.
[I 2018-05-09 10:03:22,476] Finished a trial resulted in value: 100.99613064563654. Current best value is 32.82718929591965 with parameters: {'x': -3.7295016620924066}.
[I 2018-05-09 10:03:22,477] Finished a trial resulted in value: 110.56391159932272. Current best value is 32.82718929591965 with parameters: {'x': -3.7295016620924066}.
[I 2018-05-09 10:03:22,478] Finished a trial resulted in value: 42.486606942847395. Current best value is 32.82718929591965 with parameters: {'x': -3.7295016620924066}.
[I 2018-05-09 10:03:22,479] Finished a trial resulted in value: 1.130813338091735. Current best value is 1.130813338091735 with parameters: {'x': 3.063397074517198}.
...
[I 2018-05-09 10:03:23,431] Finished a trial resulted in value: 8.760381111220335. Current best value is 0.0026232243068543526 with parameters: {'x': 1.9487825780924659}.
In [5]: study.best_params
Out[5]: {'x': 1.9487825780924659}
```

We can see that Optuna found the best `x`

value `1.9487825780924659`

, which is close to the optimal value of `2`

.

Note

In practice, it is expected that training of machine learning algorithms is invoked in objective functions, and metrics such as loss or error are reported.

## Study Object¶

Let us clarify the terminology in Optuna as follows.

**Trial**: A single call of the objective function.**Study**: An optimization session, i.e., a set of trials.**Parameter**: A variable whose value is to be optimized, e.g.,`x`

in the above example.

In Optuna, we use study object to manage optimization. Method `create_study()`

returns a study object.
A study object has useful properties to analyze the optimization outcome.

```
In [5]: study.best_params
Out[5]: {'x': 1.9926578647650126}
In [6]: study.best_value
Out[6]: 5.390694980884334e-05
In [7]: study.best_trial
Out[7]: FrozenTrial(trial_id=26, state=<TrialState.COMPLETE: 1>, params={'x': 1.9926578647650126}, user_attrs={}, system_attrs={}, value=5.390694980884334e-05, intermediate_values={}, params_in_internal_repr={'x': 1.9926578647650126}, datetime_start=datetime.datetime(2018, 5, 9, 10, 23, 0, 87060), datetime_complete=datetime.datetime(2018, 5, 9, 10, 23, 0, 91010))
In [8]: study.trials # all trials
Out[8]:
[FrozenTrial(trial_id=0, state=<TrialState.COMPLETE: 1>, params={'x': -4.219801301030433}, user_attrs={}, system_attrs={}, value=38.685928224299865, intermediate_values={}, params_in_internal_repr={'x': -4.219801301030433}, datetime_start=datetime.datetime(2018, 5, 9, 10, 22, 59, 983824), datetime_complete=datetime.datetime(2018, 5, 9, 10, 22, 59, 984053)),
...
user_attrs={}, system_attrs={}, value=8.2881000286123179, intermediate_values={}, params_in_internal_repr={'x': 4.8789060472013182}, datetime_start=datetime.datetime(2018, 5, 9, 10, 23, 0, 886434), datetime_complete=datetime.datetime(2018, 5, 9, 10, 23, 0, 891347))]
In [9]: len(study.trials)
Out[9]: 100
```

By executing `optimize()`

again, we can continue the optimization.

```
In [10]: study.optimize(objective, n_trials=100)
...
In [11]: len(study.trials)
Out[11]: 200
```