optuna.importance.FanovaImportanceEvaluator

class optuna.importance.FanovaImportanceEvaluator(*, n_trees=64, max_depth=64, seed=None)[source]

fANOVA importance evaluator.

Implements the fANOVA hyperparameter importance evaluation algorithm in An Efficient Approach for Assessing Hyperparameter Importance.

fANOVA fits a random forest regression model that predicts the objective values of COMPLETE trials given their parameter configurations. The more accurate this model is, the more reliable the importances assessed by this class are.

Note

This class takes over 1 minute when given a study that contains 1000+ trials. We published optuna-fast-fanova library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds.

Note

Requires the sklearn Python package.

Note

The performance of fANOVA depends on the prediction performance of the underlying random forest model. In order to obtain high prediction performance, it is necessary to cover a wide range of the hyperparameter search space. It is recommended to use an exploration-oriented sampler such as RandomSampler.

Note

For how to cite the original work, please refer to https://automl.github.io/fanova/cite.html.

Parameters
  • n_trees (int) – The number of trees in the forest.

  • max_depth (int) – The maximum depth of the trees in the forest.

  • seed (Optional[int]) – Controls the randomness of the forest. For deterministic behavior, specify a value other than None.

Methods

evaluate(study[, params, target])

Evaluate parameter importances based on completed trials in the given study.

evaluate(study, params=None, *, target=None)[source]

Evaluate parameter importances based on completed trials in the given study.

Note

This method is not meant to be called by library users.

See also

Please refer to get_param_importances() for how a concrete evaluator should implement this method.

Parameters
  • study (Study) – An optimized study.

  • params (Optional[List[str]]) – A list of names of parameters to assess. If None, all parameters that are present in all of the completed trials are assessed.

  • target (Optional[Callable[[FrozenTrial], float]]) –

    A function to specify the value to evaluate importances. If it is None and study is being used for single-objective optimization, the objective values are used. Can also be used for other trial attributes, such as the duration, like target=lambda t: t.duration.total_seconds().

    Note

    Specify this argument if study is being used for multi-objective optimization. For example, to get the hyperparameter importance of the first objective, use target=lambda t: t.values[0] for the target parameter.

Returns

An collections.OrderedDict where the keys are parameter names and the values are assessed importances.

Return type

Dict[str, float]