optuna.integration.TorchDistributedTrial

class optuna.integration.TorchDistributedTrial(trial, group=None)[source]

A wrapper of Trial to incorporate Optuna with PyTorch distributed.

See also

TorchDistributedTrial provides the same interface as Trial. Please refer to optuna.trial.Trial for further details.

See the example if you want to optimize an objective function that trains neural network written with PyTorch distributed data parallel.

Parameters

trial (Optional[Trial]) – A Trial object or None. Please set trial object in rank-0 node and set None in the other rank node.
group (Optional[ProcessGroup]) –
A torch.distributed.ProcessGroup to communicate with the other nodes. TorchDistributedTrial use CPU tensors to communicate, make sure the group supports CPU tensors communications.

Use gloo backend when group is None. Create a global gloo backend when group is None and WORLD is nccl.

Note

The methods of TorchDistributedTrial are expected to be called by all workers at once. They invoke synchronous data transmission to share processing results and synchronize timing.

Note

Added in v2.6.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v2.6.0.

Methods

`report`(value, step)
`set_system_attr`(key, value)
`set_user_attr`(key, value)
`should_prune`()
`suggest_categorical`()
`suggest_discrete_uniform`(name, low, high, q)
`suggest_float`(name, low, high, *[, step, log])
`suggest_int`(name, low, high[, step, log])
`suggest_loguniform`(name, low, high)
`suggest_uniform`(name, low, high)

Attributes

`datetime_start`
`distributions`
`number`
`params`
`system_attrs`
`user_attrs`

set_system_attr(key, value)[source]

Warning

Deprecated in v3.1.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. See https://github.com/optuna/optuna/releases/tag/v3.1.0.

Parameters

key (str) –
value (Any) –

Return type

None

suggest_discrete_uniform(name, low, high, q)[source]

Warning

Deprecated in v3.0.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. See https://github.com/optuna/optuna/releases/tag/v3.0.0.

Use suggest_float() instead.

Parameters

name (str) –
low (float) –
high (float) –
q (float) –

Return type

float

suggest_loguniform(name, low, high)[source]

Warning

Deprecated in v3.0.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. See https://github.com/optuna/optuna/releases/tag/v3.0.0.

Use suggest_float() instead.

Parameters

name (str) –
low (float) –
high (float) –

Return type

float

suggest_uniform(name, low, high)[source]

Warning

Deprecated in v3.0.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. See https://github.com/optuna/optuna/releases/tag/v3.0.0.

Use suggest_float() instead.

Parameters

name (str) –
low (float) –
high (float) –

Return type

float

property system_attrs: Dict[str, Any]: Warning

Deprecated in v3.1.0. This feature will be removed in the future. The removal of this feature is currently scheduled for v6.0.0, but this schedule is subject to change. See https://github.com/optuna/optuna/releases/tag/v3.1.0.