optuna.importance.PedAnovaImportanceEvaluator

class optuna.importance.PedAnovaImportanceEvaluator(*, target_quantile=0.1, region_quantile=1.0, baseline_quantile=None, evaluate_on_local=True)[source]

PED-ANOVA importance evaluator.

Implements the PED-ANOVA hyperparameter importance evaluation algorithm.

PED-ANOVA fits Parzen estimators of COMPLETE trials better than a user-specified target_quantile. The importance can be interpreted as how important each hyperparameter is to get the performance better than target_quantile.

For further information about PED-ANOVA algorithm, please refer to the following paper:

target_quantile and region_quantile correspond to the parameters gamma' and gamma in the original paper, respectively.

Note

The performance of PED-ANOVA depends on how many trials to consider above target_quantile. To stabilize the analysis, it is preferable to include at least 5 trials above target_quantile.

Note

Please refer to the original work.

Parameters:

  • target_quantile (float) – Compute the importance of achieving top-target_quantile quantile objective value. For example, target_quantile=0.1 means that the importances give the information of which parameters were important to achieve the top-10% performance during optimization.

  • region_quantile (float) – Define the region where we compute the importance. For example, region_quantile=0.5 means that we compute the importance in the region where trials achieve top-50% performance. If region_quantile=1.0, the importance is computed in the whole search space.

  • baseline_quantile (float | None) –

    Compute the importance of achieving top-baseline_quantile quantile objective value. For example, baseline_quantile=0.1 means that the importances give the information of which parameters were important to achieve the top-10% performance during optimization.

    Warning

    Deprecated in v4.7.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v5.0.0, but this schedule is subject to change. baseline_quantile is currently ignored. Use target_quantile instead. See https://github.com/optuna/optuna/releases/tag/v4.7.0.

  • evaluate_on_local (bool) – Whether we measure the importance in the local or global space. If True, the importances imply how importance each parameter is during optimization. Meanwhile, evaluate_on_local=False gives the importances in the specified search_space. evaluate_on_local=True is especially useful when users modify search space during optimization.

Example

An example of using PED-ANOVA is as follows:

import optuna
from optuna.importance import PedAnovaImportanceEvaluator


def objective(trial):
    x1 = trial.suggest_float("x1", -10, 10)
    x2 = trial.suggest_float("x2", -10, 10)
    return x1 + x2 / 1000


study = optuna.create_study()
study.optimize(objective, n_trials=100)
evaluator = PedAnovaImportanceEvaluator()
importance = optuna.importance.get_param_importances(study, evaluator=evaluator)

Note

Added in v3.6.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v3.6.0.

Methods

evaluate(study[, params, target])

Evaluate parameter importances based on completed trials in the given study.
evaluate(study, params=None, *, target=None)[source]

Evaluate parameter importances based on completed trials in the given study.

Note

This method is not meant to be called by library users.

See also

Please refer to get_param_importances() for how a concrete evaluator should implement this method.

Parameters:

  • study (Study) – An optimized study.

  • params (list[str] | None) – A list of names of parameters to assess. If None, all parameters that are present in all of the completed trials are assessed.

  • target (Callable[[FrozenTrial], float] | None) –

    A function to specify the value to evaluate importances. If it is None and study is being used for single-objective optimization, the objective values are used. Can also be used for other trial attributes, such as the duration, like target=lambda t: t.duration.total_seconds().

    Note

    Specify this argument if study is being used for multi-objective optimization. For example, to get the hyperparameter importance of the first objective, use target=lambda t: t.values[0] for the target parameter.

Returns:

A dict where the keys are parameter names and the values are assessed importances.

Return type:

dict[str, float]