ExperimentRuns

class verta.tracking.entities.ExperimentRuns(conn, conf)

list-like object representing a collection of machine learning Experiment Runs.

This class provides functionality for filtering and sorting its contents.

There should not be a need to instantiate this class directly; please use other classes’ attributes to access Experiment Runs.

Examples

runs = expt.find("hyperparameters.hidden_size == 256")
len(runs)
# 12
runs = runs.find("metrics.accuracy >= .8")
len(runs)
# 5
runs[0].get_metric("accuracy")
# 0.8921755939794525

# find runs by a tag:
runs_with_tag = runs.find("tags ~= some-tag")
as_dataframe()

Returns this collection of Experiment Runs as a table.

Returns:

pandas.DataFrame

bottom_k(key, k, ret_all_info=False)

Gets the Experiment Runs from this collection with the k lowest keys.

Changed in version 0.14.12: The ret_all_info parameter was removed.

A key is a string containing a dot-delimited Experiment Run property such as metrics.accuracy.

Parameters:

  • key (str) – Dot-delimited Experiment Run property.

  • k (int) – Number of Experiment Runs to get.

Returns:

ExperimentRuns

Warning

This feature is still in active development. It is completely safe to use, but may exhibit unintuitive behavior. Please report any oddities to the Verta team!

Examples

runs.bottom_k("metrics.loss", 3)
# <ExperimentRuns containing 3 runs>
find(*args)

Gets the results from this collection that match input predicates.

A predicate is a string containing a simple boolean expression consisting of:

  • a dot-delimited property such as metrics.accuracy

  • a Python boolean operator such as >=

  • a literal value such as .8

Parameters:

*args (strs) – Predicates specifying results to get.

Returns:

The same type of object given in the input.

Examples

runs.find("hyperparameters.hidden_size == 256",
           "metrics.accuracy >= .8")
# <ExperimentRuns containing 3 runs>
# alternatively:
runs.find(["hyperparameters.hidden_size == 256",
           "metrics.accuracy >= .8"])
# <ExperimentRuns containing 3 runs>
set_page_limit(limit)

Sets the number of entities to fetch per backend call during iteration.

By default, each call fetches a batch of 100 entities, but lowering this value may be useful for substantially larger responses.

Parameters:

limit (int) – Number of entities to fetch per call.

Examples

runs = proj.expt_runs
runs.set_page_limit(10)
for run in runs:  # fetches 10 runs per backend call
    print(run.get_metric("accuracy"))
sort(key, descending=False)

Sorts the results from this collection by key.

A key is a string containing a dot-delimited property such as metrics.accuracy.

Parameters:

  • key (str) – Dot-delimited property.

  • descending (bool, default False) – Order in which to return sorted results.

Returns:

The same type of object given in the input.

Examples

runs.sort("metrics.accuracy")
# <ExperimentRuns containing 3 runs>
top_k(key, k, ret_all_info=False)

Gets the Experiment Runs from this collection with the k highest keys.

Changed in version 0.14.12: The ret_all_info parameter was removed.

A key is a string containing a dot-delimited Experiment Run property such as metrics.accuracy.

Parameters:

  • key (str) – Dot-delimited Experiment Run property.

  • k (int) – Number of Experiment Runs to get.

Returns:

ExperimentRuns

Warning

This feature is still in active development. It is completely safe to use, but may exhibit unintuitive behavior. Please report any oddities to the Verta team!

Examples

runs.top_k("metrics.accuracy", 3)
# <ExperimentRuns containing 3 runs>
with_workspace(workspace=None)

Returns experiment runs in the specified workspace.

Parameters:

workspace (str, optional) – Workspace name. If not provided, uses personal workspace.

Returns:

ExperimentRuns – Filtered experiment runs.