ray.air.integrations.comet.CometLoggerCallback#

class ray.air.integrations.comet.CometLoggerCallback(online: bool = True, tags: Optional[List[str]] = None, save_checkpoints: bool = False, **experiment_kwargs)[source]#

Bases: ray.tune.logger.logger.LoggerCallback

CometLoggerCallback for logging Tune results to Comet.

Comet (https://comet.ml/site/) is a tool to manage and optimize the entire ML lifecycle, from experiment tracking, model optimization and dataset versioning to model production monitoring.

This Ray Tune LoggerCallback sends metrics and parameters to Comet for tracking.

In order to use the CometLoggerCallback you must first install Comet via pip install comet_ml

Then set the following environment variables export COMET_API_KEY=<Your API Key>

Alternatively, you can also pass in your API Key as an argument to the CometLoggerCallback constructor.

CometLoggerCallback(api_key=<Your API Key>)

Parameters

  • online – Whether to make use of an Online or Offline Experiment. Defaults to True.

  • tags – Tags to add to the logged Experiment. Defaults to None.

  • save_checkpoints – If True, model checkpoints will be saved to Comet ML as artifacts. Defaults to False.

  • **experiment_kwargs – Other keyword arguments will be passed to the constructor for comet_ml.Experiment (or OfflineExperiment if online=False).

Please consult the Comet ML documentation for more information on the Experiment and OfflineExperiment classes: https://comet.ml/site/

Example:

from ray.air.integrations.comet import CometLoggerCallback
tune.run(
    train,
    config=config
    callbacks=[CometLoggerCallback(
        True,
        ['tag1', 'tag2'],
        workspace='my_workspace',
        project_name='my_project_name'
        )]
)

Methods

get_state()

Get the state of the callback.

log_trial_restore(trial)

Handle logging when a trial restores.

log_trial_result(iteration, trial, result)

Log the current result of a Trial upon each iteration.

log_trial_start(trial)

Initialize an Experiment (or OfflineExperiment if self.online=False) and start logging to Comet.

on_checkpoint(iteration, trials, trial, ...)

Called after a trial saved a checkpoint with Tune.

on_experiment_end(trials, **info)

Called after experiment is over and all trials have concluded.

on_step_begin(iteration, trials, **info)

Called at the start of each tuning loop step.

on_step_end(iteration, trials, **info)

Called at the end of each tuning loop step.

on_trial_recover(iteration, trials, trial, ...)

Called after a trial instance failed (errored) but the trial is scheduled for retry.

set_state(state)

Set the state of the callback.

setup([stop, num_samples, total_num_samples])

Called once at the very beginning of training.