{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n\n# User-Defined Sampler\n\nThanks to user-defined samplers, you can:\n\n- experiment your own sampling algorithms,\n- implement task-specific algorithms to refine the optimization performance, or\n- wrap other optimization libraries to integrate them into Optuna pipelines (e.g., [BoTorchSampler](https://optuna-integration.readthedocs.io/en/stable/reference/generated/optuna_integration.BoTorchSampler.html)).\n\nThis section describes the internal behavior of sampler classes and shows an example of implementing a user-defined sampler.\n\n\n## Overview of Sampler\n\nA sampler has the responsibility to determine the parameter values to be evaluated in a trial.\nWhen a `suggest` API (e.g., :func:`~optuna.trial.Trial.suggest_float`) is called inside an objective function, the corresponding distribution object (e.g., :class:`~optuna.distributions.FloatDistribution`) is created internally. A sampler samples a parameter value from the distribution. The sampled value is returned to the caller of the `suggest` API and evaluated in the objective function.\n\nTo create a new sampler, you need to define a class that inherits :class:`~optuna.samplers.BaseSampler`.\nThe base class has three abstract methods;\n:meth:`~optuna.samplers.BaseSampler.infer_relative_search_space`,\n:meth:`~optuna.samplers.BaseSampler.sample_relative`, and\n:meth:`~optuna.samplers.BaseSampler.sample_independent`.\n\nAs the method names imply, Optuna supports two types of sampling: one is **relative sampling** that can consider the correlation of the parameters in a trial, and the other is **independent sampling** that samples each parameter independently.\n\nAt the beginning of a trial, :meth:`~optuna.samplers.BaseSampler.infer_relative_search_space` is called to provide the relative search space for the trial. Then, :meth:`~optuna.samplers.BaseSampler.sample_relative` is invoked to sample relative parameters from the search space. During the execution of the objective function, :meth:`~optuna.samplers.BaseSampler.sample_independent` is used to sample parameters that don't belong to the relative search space.\n\n
Please refer to the document of :class:`~optuna.samplers.BaseSampler` for further details.
In favor of code simplicity, the above implementation doesn't support some features (e.g., maximization).\n If you're interested in how to support those features, please see\n [examples/samplers/simulated_annealing.py](https://github.com/optuna/optuna-examples/blob/main/samplers/simulated_annealing_sampler.py).
Strictly speaking, in the first trial,\n ``SimulatedAnnealingSampler.sample_independent`` method is used to sample parameter values.\n Because :func:`~optuna.search_space.intersection_search_space` used in\n ``SimulatedAnnealingSampler.infer_relative_search_space`` cannot infer the search space\n if there are no complete trials.