optuna.terminator.EMMREvaluator
- class optuna.terminator.EMMREvaluator(deterministic_objective=False, delta=0.1, min_n_trials=2, seed=None)[source]
Evaluates a kind of regrets, called the Expected Minimum Model Regret(EMMR).
EMMR is an upper bound of “expected minimum simple regret” in the optimization process.
Expected minimum simple regret is a quantity that converges to zero only if the optimization process has found the global optima.
For further information about expected minimum simple regret and the algorithm, please refer to the following paper:
Also, there is our blog post explaining this evaluator:
- Parameters:
deterministic_objective (bool) – A boolean value which indicates whether the objective function is deterministic. Default is
False
.delta (float) – A float number related to the criterion for termination. Default to 0.1. For further information about this parameter, please see the aforementioned paper.
min_n_trials (int) – A minimum number of complete trials to compute the criterion. Default to 2.
seed (int | None) – A random seed for EMMREvaluator.
Example
import optuna from optuna.terminator import EMMREvaluator from optuna.terminator import MedianErrorEvaluator from optuna.terminator import Terminator sampler = optuna.samplers.TPESampler(seed=0) study = optuna.create_study(sampler=sampler, direction="minimize") emmr_improvement_evaluator = EMMREvaluator() median_error_evaluator = MedianErrorEvaluator(emmr_improvement_evaluator) terminator = Terminator( improvement_evaluator=emmr_improvement_evaluator, error_evaluator=median_error_evaluator, ) for i in range(1000): trial = study.ask() ys = [trial.suggest_float(f"x{i}", -10.0, 10.0) for i in range(5)] value = sum(ys[i] ** 2 for i in range(5)) study.tell(trial, value) if terminator.should_terminate(study): # Terminated by Optuna Terminator! break
Note
Added in v4.0.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v4.0.0.
Methods
evaluate
(trials, study_direction)