optuna.importance.PedAnovaImportanceEvaluator

class optuna.importance.PedAnovaImportanceEvaluator(*, target_quantile=0.1, region_quantile=1.0, evaluate_on_local=True)[source]

PED-ANOVA importance evaluator.

Implements the PED-ANOVA hyperparameter importance evaluation algorithm.

PED-ANOVA fits Parzen estimators of COMPLETE trials better than a user-specified target_quantile. The importance can be interpreted as how important each hyperparameter is to get the performance better than target_quantile.

For further information about PED-ANOVA algorithm, please refer to the following paper:

PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces (IJCAI 2023)

For further information on how conditional parameters are handled, please refer to the following paper:

Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces (KDD 2026)

target_quantile and region_quantile correspond to the parameters \(\gamma'\) and \(\gamma\) in the original paper, respectively.

Note

Behavior on multi-objective studies. If target is None, top-quantile trials are selected in the same manner as multi-objective TPESampler: trials are ranked by non-domination rank, with the hypervolume subset selection problem (HSSP) used to break ties within a rank. The resulting importance can be interpreted as how important each hyperparameter is to reach the Pareto front without preference for any particular objective.

To compute the importance against a single objective instead, pass a target callable explicitly. Note that PedAnovaImportanceEvaluator assumes minimization (i.e., lower target values are better); when an objective is being maximized, negate it inside target:

# Objective 0 is being minimized.
importance = get_param_importances(
    study,
    evaluator=PedAnovaImportanceEvaluator(),
    target=lambda t: t.values[0],
)

# Objective 0 is being maximized—negate so that "lower is better".
importance = get_param_importances(
    study,
    evaluator=PedAnovaImportanceEvaluator(),
    target=lambda t: -t.values[0],
)

Note

The performance of PED-ANOVA depends on how many trials to consider above target_quantile. To stabilize the analysis, it is preferable to include at least 5 trials above target_quantile.

Note

Please also refer to the original implementations:

Parameters:

target_quantile (float) – Compute the importance of achieving top-target_quantile quantile objective value. For example, target_quantile=0.1 means that the importances give the information of which parameters were important to achieve the top-10% performance during optimization.
region_quantile (float) – Define the region where we compute the importance. For example, region_quantile=0.5 means that we compute the importance in the region where trials achieve top-50% performance. If region_quantile=1.0, the importance is computed in the whole search space.
evaluate_on_local (bool) – Whether we measure the importance in the local or global space. If True, the importances imply how importance each parameter is during optimization. Meanwhile, evaluate_on_local=False gives the importances in the specified search_space. evaluate_on_local=True is especially useful when users modify search space during optimization.

Example

An example of using PED-ANOVA is as follows:

import optuna
from optuna.importance import PedAnovaImportanceEvaluator


def objective(trial):
    x1 = trial.suggest_float("x1", -10, 10)
    x2 = trial.suggest_float("x2", -10, 10)
    return x1 + x2 / 1000


study = optuna.create_study()
study.optimize(objective, n_trials=100)
evaluator = PedAnovaImportanceEvaluator()
importance = optuna.importance.get_param_importances(study, evaluator=evaluator)

Methods

evaluate(study[, params, target])

Evaluate parameter importances based on completed trials in the given study.

evaluate(study, params=None, *, target=None)[source]

Evaluate parameter importances based on completed trials in the given study.

Note

This method is not meant to be called by library users.