optuna.importance.FanovaImportanceEvaluator
- class optuna.importance.FanovaImportanceEvaluator(*, n_trees=64, max_depth=64, seed=None)[source]
fANOVA importance evaluator.
Implements the fANOVA hyperparameter importance evaluation algorithm in An Efficient Approach for Assessing Hyperparameter Importance.
fANOVA fits a random forest regression model that predicts the objective values of
COMPLETE
trials given their parameter configurations. The more accurate this model is, the more reliable the importances assessed by this class are.Note
This class takes over 1 minute when given a study that contains 1000+ trials. We published optuna-fast-fanova library, that is a Cython accelerated fANOVA implementation. By using it, you can get hyperparameter importances within a few seconds.
Note
Requires the sklearn Python package.
Note
The performance of fANOVA depends on the prediction performance of the underlying random forest model. In order to obtain high prediction performance, it is necessary to cover a wide range of the hyperparameter search space. It is recommended to use an exploration-oriented sampler such as
RandomSampler
.Note
For how to cite the original work, please refer to https://automl.github.io/fanova/cite.html.
- Parameters:
Methods
evaluate
(study[, params, target])Evaluate parameter importances based on completed trials in the given study.
- evaluate(study, params=None, *, target=None)[source]
Evaluate parameter importances based on completed trials in the given study.
Note
This method is not meant to be called by library users.
See also
Please refer to
get_param_importances()
for how a concrete evaluator should implement this method.- Parameters:
study (Study) – An optimized study.
params (list[str] | None) – A list of names of parameters to assess. If
None
, all parameters that are present in all of the completed trials are assessed.target (Callable[[FrozenTrial], float] | None) –
A function to specify the value to evaluate importances. If it is
None
andstudy
is being used for single-objective optimization, the objective values are used. Can also be used for other trial attributes, such as the duration, liketarget=lambda t: t.duration.total_seconds()
.Note
Specify this argument if
study
is being used for multi-objective optimization. For example, to get the hyperparameter importance of the first objective, usetarget=lambda t: t.values[0]
for the target parameter.
- Returns:
A
dict
where the keys are parameter names and the values are assessed importances.- Return type: