optuna.artifacts

The artifacts module provides the way to manage artifacts (output files) in Optuna. Please also check Optuna Artifacts Tutorial and our article. The storages covered by artifacts are the following:

Class Name

Supported Storage

FileSystemArtifactStore

Local File System, Network File System

Boto3ArtifactStore

Amazon S3 Compatible Object Storage

GCSArtifactStore

Google Cloud Storage

Note

The methods defined in each ArtifactStore are not intended to be directly accessed by library users.

Note

As ArtifactStore does not officially provide user API for artifact removal, please refer to How can I delete all the artifacts uploaded to a study? for the hack.

class optuna.artifacts.FileSystemArtifactStore(base_path)[source]

An artifact store for file systems.

Parameters:

base_path (str | Path) – The base path to a directory to store artifacts.

Example

import os

import optuna
from optuna.artifacts import FileSystemArtifactStore
from optuna.artifacts import upload_artifact


base_path = "./artifacts"
os.makedirs(base_path, exist_ok=True)
artifact_store = FileSystemArtifactStore(base_path=base_path)


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...
class optuna.artifacts.Boto3ArtifactStore(bucket_name, client=None, *, avoid_buf_copy=False)[source]

An artifact backend for Boto3.

Parameters:

  • bucket_name (str) – The name of the bucket to store artifacts.

  • client (S3Client | None) – A Boto3 client to use for storage operations. If not specified, a new client will be created.

  • avoid_buf_copy (bool) – If True, skip procedure to copy the content of the source file object to a buffer before uploading it to S3 ins. This is default to False because using upload_fileobj() method of Boto3 client might close the source file object.

Example

import optuna
from optuna.artifacts import upload_artifact
from optuna.artifacts import Boto3ArtifactStore


artifact_store = Boto3ArtifactStore("my-bucket")


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...
class optuna.artifacts.GCSArtifactStore(bucket_name, client=None)[source]

An artifact backend for Google Cloud Storage (GCS).

Parameters:

  • bucket_name (str) – The name of the bucket to store artifacts.

  • client (google.cloud.storage.Client | None) – A google-cloud-storage Client to use for storage operations. If not specified, a new client will be created with default settings.

Example

import optuna
from optuna.artifacts import GCSArtifactStore, upload_artifact


artifact_backend = GCSArtifactStore("my-bucket")


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...

Before running this code, you will have to install gcloud and run

gcloud auth application-default login

so that the Cloud Storage library can automatically find the credential.

Note

Added in v3.4.0 as an experimental feature. The interface may change in newer versions without prior notice. See https://github.com/optuna/optuna/releases/tag/v3.4.0.

class optuna.artifacts.Backoff(backend, *, max_retries=10, multiplier=2, min_delay=0.1, max_delay=30)[source]

An artifact store’s middleware for exponential backoff.

Example

import optuna
from optuna.artifacts import upload_artifact
from optuna.artifacts import Boto3ArtifactStore
from optuna.artifacts import Backoff


artifact_store = Backoff(Boto3ArtifactStore("my-bucket"))


def objective(trial: optuna.Trial) -> float:
    ... = trial.suggest_float("x", -10, 10)
    file_path = generate_example(...)
    upload_artifact(
        artifact_store=artifact_store,
        file_path=file_path,
        study_or_trial=trial,
    )
    return ...
Parameters:

  • backend (ArtifactStore)

  • max_retries (int)

  • multiplier (float)

  • min_delay (float)

  • max_delay (float)

class optuna.artifacts.ArtifactMeta(artifact_id, filename, mimetype, encoding)[source]

Meta information for an artifact.

Note

All the artifact meta linked to a study or trial can be listed by get_all_artifact_meta(). The artifact meta can be used for download_artifact().

Parameters:

  • artifact_id (str) – The identifier of the artifact.

  • filename (str) – The artifact file name used for the upload.

  • mimetype (str) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension.

  • encoding (str | None) – An encoding of the artifact, which is suitable for use as a Content-Encoding header, e.g., gzip. If not specified, the encoding is guessed from the file extension.

optuna.artifacts.upload_artifact(*, artifact_store, file_path, study_or_trial, storage=None, mimetype=None, encoding=None)[source]

Upload an artifact to the artifact store.

Parameters:

  • artifact_store (ArtifactStore) – An artifact store.

  • file_path (str) – A path to the file to be uploaded.

  • study_or_trial (Trial | FrozenTrial | Study) – A Trial object, a FrozenTrial, or a Study object.

  • storage (BaseStorage | None) – A storage object. This argument is required only if study_or_trial is FrozenTrial.

  • mimetype (str | None) – A MIME type of the artifact. If not specified, the MIME type is guessed from the file extension.

  • encoding (str | None) – An encoding of the artifact, which is suitable for use as a Content-Encoding header (e.g. gzip). If not specified, the encoding is guessed from the file extension.

Returns:

An artifact ID.

Return type:

str

optuna.artifacts.get_all_artifact_meta(study_or_trial, *, storage=None)[source]

List the associated artifact information of the provided trial or study.

Parameters:
Return type:

list[ArtifactMeta]

Example

An example where this function is useful:

import os

import optuna


# Get the storage that contains the study of interest.
storage = optuna.storages.get_storage(storage=...)

# Instantiate the artifact store used for the study.
# Optuna does not provide the API that stores the used artifact store information, so
# please manage the information in the user side.
artifact_store = ...

# Load study that contains the artifacts of interest.
study = optuna.load_study(study_name=..., storage=storage)

# Fetch the best trial.
best_trial = study.best_trial

# Fetch all the artifact meta connected to the best trial.
artifact_metas = optuna.artifacts.get_all_artifact_meta(best_trial, storage=storage)

download_dir_path = "./best_trial_artifacts/"
os.makedirs(download_dir_path, exist_ok=True)

for artifact_meta in artifact_metas:
    download_file_path = os.path.join(download_dir_path, artifact_meta.filename)
    # Download the artifacts to ``download_file_path``.
    optuna.artifacts.download_artifact(
        artifact_store=artifact_store,
        artifact_id=artifact_meta.artifact_id,
        file_path=download_file_path,
    )
Returns:

The list of artifact meta in the trial or study. Each artifact meta includes artifact_id, filename, mimetype, and encoding. Note that if Study is provided, we return the information of the artifacts uploaded to study, but not to all the trials in the study.

Parameters:
Return type:

list[ArtifactMeta]

optuna.artifacts.download_artifact(*, artifact_store, file_path, artifact_id)[source]

Download an artifact from the artifact store.

Parameters:

  • artifact_store (ArtifactStore) – An artifact store.

  • file_path (str) – A path to save the downloaded artifact.

  • artifact_id (str) – The identifier of the artifact to download.

Return type:

None