Getting Started
API ReferenceSpell CLI Documentation
Welcome to the spell command line interface documentation. Below you will find a description of each command, including the various options and arguments.
spell
Usage: spell [OPTIONS] COMMAND [ARGS]...
Options:
-h, --help Show this message and exit.
--version Show the version and exit.
Commands:
archive Specify one or more Run IDs to archive
cluster Manage external clusters
cp Retrieve a file or directory
feedback Provide feedback to the Spell team
help Display help information
hyper Create hyperparameter searches
info Describes a run. Displays info such as start and end time as well as
run parameters such as apts, pips, and mounts.
jupyter Create a jupyter workspace
keys Manage public SSH keys registered with Spell
kill Kill a current run.
label Adds and remove labels from a run
link List symlinks, find symlinks by alias, or create a symlink to a
resource path.
login Log in with your username or email
logout Log out of current session
logs Retrieve logs for a run
ls List resource files
model-servers Manage model servers
passwd Set a new password for your Spell account
ps Display run statuses
repos List repositories
rm Specify one or more uploaded resources to delete. For example:
uploads/[directory]
run Execute a new run
server Manage model servers
stats Display performance statistics for a run
status Display account and billing information
stop Stop a run with status 'Running'
unlink Remove a link.
upload Upload a file or directory
whoami Display current user information
workflow Execute a new workflow
spell archive
Usage: spell archive [OPTIONS] RUN_IDS...
Archive one or more runs. Use to remove a finished or failed run by specifying its
RUN_ID.
The removed runs will no longer show up in `ps`. The outputs of removed runs and
removed uploads will no longer appear in `ls` or be mountable on another run with
`--mount`.
Options:
-q, --quiet Suppress logging
--help Show this message and exit.
spell cluster
Usage: spell cluster [OPTIONS] COMMAND [ARGS]...
Manage external clusters on Spell
With no subcommand, display all your external clusters
Options:
--help Show this message and exit.
Commands:
add-bucket Adds a cloud storage bucket to SpellFS
add-docker-registry Configures your cluster to enable runs with docker images in
the private registry hosted by your cloud provider (ECR or GCR
respectively)
create-kube-cluster Sets up a GKE/EKS kubernetes cluster in your Spell VPC.
Required for model serving.
delete Deletes a given cluster
delete-docker-registry Removes your cluster's access to docker images in the private
registry hosted by your cloud provider (ECR or GCR
respectively).
init Create a cluster
machine-type Manage machine types
update Makes sure your Spell cluster is fully up to date and able to
support the latest features
spell cluster add-bucket
Usage: spell cluster add-bucket [OPTIONS]
This command adds a cloud storage bucket (S3 or GS) to SpellFS, which enables
interaction with the bucket objects via ls, cp, and mounts. It will also updates the
permissions of that bucket to allow Spell read access to it
Options:
-p, --profile TEXT This AWS profile will be used to get your Access Key ID and Secret
as well as your Region. You will be prompted to confirm the Key
and Region are correct before continuing. This key will be used to
adjust IAM permissions of the role associated with the cluster
that the bucket is being added to.
--bucket TEXT Name of bucket
--help Show this message and exit.
spell cluster add-docker-registry
Usage: spell cluster add-docker-registry [OPTIONS]
This command enables pulling docker images from a private registry. Read permissions
to the registry will be added to the IAM role associated with the cluster.
Options:
--cluster-name TEXT Name of cluster to add registry permissions to
--repo TEXT Name of repository. ECR only
-p, --profile TEXT This AWS profile will be used to get your Access Key ID and
Secret as well as your Region. You will be prompted to confirm
the Key and Region are correct before continuing. This key will
be used to adjust IAM permissions of the role associated with the
cluster that needs access to the registry.
--help Show this message and exit.
spell cluster create-kube-cluster
Usage: spell cluster create-kube-cluster [OPTIONS]
Sets up a GKE or EKS kubernetes cluster in your Spell VPC. This cluster is required
for model serving. Spell will automatically create a CPU node group for you which
will have at least one machine running at all times.
Options:
-p, --profile TEXT AWS profile to pull credentials from
--nodes-min INTEGER Minimum number of nodes in the model serving cluster
(default 1)
--nodes-max INTEGER Minimum number of nodes in the model serving cluster
(default 2)
--node-disk-size INTEGER Size of disks on each node in GB (default 50GB)
--use-existing This is an advanced option to use an existing EKS/GKE
cluster instead of creating a new one.
Because Spell sets up
your cluster in a particular manner this option is only
likely to work with clusters created
exactly the way we set
ours up. This flag is likely only valuable if you
experienced an error the first
time you tried to run this
command, but the kube cluster creation succeeded.
--aws-zones TEXT Allows AWS clusters to explicitly list the availability
zones used for the EKS cluster. List the desired AZs as
comma separated values, ex: 'us-east-1a,us-east-1c,us-
east-1d'. NOTE: Most users will NOT have to do this. This is
useful if there are problems with one or more of the AZs in
the region of your cluster.
--from-file FILENAME A path to a YAML or JSON file to load the arguments and
options from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell cluster delete
Usage: spell cluster delete [OPTIONS]
Facilitates the deletion of your Spell cluster by removing the associated
infrastructure on Spell as well as deleting all associated cloud resources. It will
OPTIONALLY delete the data in your output bucket - including run outputs.
Options:
-p, --profile TEXT This AWS profile will be used to get your Access Key ID and Secret
as well as your Region. You will be prompted to confirm the Key
and Region are correct before continuing. This key will be used to
destroy the VPC, IAM Roles, and optionally the S3 bucket created
for the cluster.
--help Show this message and exit.
spell cluster delete-docker-registry
Usage: spell cluster delete-docker-registry [OPTIONS]
This command removes your cluster's access to docker images in the private registry
hosted by your cloud provider.
Options:
--repo TEXT Name of repository. ECR only
--cluster-name TEXT Name of cluster to remove registry permissions from
-p, --profile TEXT This AWS profile will be used to get your Access Key ID and
Secret as well as your Region. You will be prompted to confirm
the Key and Region are correct before continuing. This key will
be used to adjust IAM permissions of the role associated with the
cluster that has access to the registry.
--help Show this message and exit.
spell cluster init
Usage: spell cluster init [OPTIONS] COMMAND [ARGS]...
Create a new aws/gcp cluster for your org account
Set up a cluster to use machines in your own AWS/GCP account
Options:
--help Show this message and exit.
Commands:
aws Sets up an AWS VPC as a Spell cluster
gcp Sets up GCP VPC as a Spell cluster
spell cluster init aws
Usage: spell cluster init aws [OPTIONS]
This command sets an AWS VPC of your choosing as an external Spell cluster. This
will let your organization run runs in that VPC, so your data never leaves your VPC.
You set an S3 bucket of your choosing for all run outputs to be written to. After
this cluster is set up you will be able to select the types and number of machines
you would like Spell to create in this cluster.
NOTE: This command uses your AWS credentials, found in ~/.aws/credentials to create
the necessary AWS resources for Spell to access and manage those machines. Your AWS
credentials will need permission to setup these resources.
Options:
-n, --name TEXT This will be used by Spell for you to identify the cluster
-p, --profile TEXT This AWS profile will be used to get your Access Key ID and Secret
as well as your Region. You will be prompted to confirm the Key
and Region are correct before continuing. This key will be used to
create all the resources necessary for Spell to manage machines in
your external VPC. It must have permissions to create these
resources.
--help Show this message and exit.
spell cluster init gcp
Usage: spell cluster init gcp [OPTIONS]
This command creates a Spell cluster within a GCP VPC of your choosing as an
external Spell cluster. This will let your organization run runs in that VPC, so
your data never leaves your VPC. You set an GCS bucket of your choosing for all run
outputs to be written to. After this cluster is set up you will be able to select
the types and number of machines you would like Spell to create in this cluster.
NOTE: This command uses your GCP credentials, activated by running `gcloud auth
application-default login`, to create the necessary GCP resources for Spell to
access and manage those machines. Your GCP credentials will need permission to set
up these resources.
Options:
-n, --name TEXT Name used by Spell for you to identify this GCP cluster
--help Show this message and exit.
spell cluster machine-type
Usage: spell cluster machine-type [OPTIONS] COMMAND [ARGS]...
Manage groups of similar machines which can be used for training runs and workspaces
on Spell
With no subcommand, display all your machine types
Options:
--help Show this message and exit.
Commands:
add Creates a new machine type for executing Spell Runs and Workspaces
delete Delete a machine type
get-token Delete a machine type
scale Change the limits for number of machines of this machine type
spell cluster machine-type add
Usage: spell cluster machine-type add [OPTIONS]
Options:
--name TEXT Name to give this machine type
--instance-type TEXT The type of machine to use e.g. 'CPU', 'K80'. If you
skip this you will be prompted with options
--spot Spot/Preemptible instances can be significantly
cheaper than on demand instances, however AWS/GCP can
terminate them at any time. If your run is terminated
prematurely we will keep all data and save it for you
with a final run status of Interrupted.
--storage-size INTEGER Disk size in GB
--additional-images [DyNet|Torch (Lua)|Fast.ai|Caffe|TensorFlow 2|MXNet]
By default all machines support TensorFlow 1, PyTorch,
and Conda. If you require any additional frameworks
(e.g. Caffe, MXNet) you can select them here.
Additional frameworks will increase the time it takes
to spin up new machines.
--min-machines INTEGER Minimum number of machines to keep available at all
times regardless of demand
--max-machines INTEGER Maximum number of machines of this machine type
--idle-timeout INTEGER Grace period to wait before terminating idle machines
(minutes)
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell cluster machine-type delete
Usage: spell cluster machine-type delete [OPTIONS] NAME
Options:
-f, --force Do not prompt for confirmation
--help Show this message and exit.
spell cluster machine-type get-token
Usage: spell cluster machine-type get-token [OPTIONS] NAME
Options:
--renew Renew token. This will invalidate the previous token
--help Show this message and exit.
spell cluster machine-type scale
Usage: spell cluster machine-type scale [OPTIONS] NAME
Options:
--min-machines INTEGER Minimum number of machines to keep available at all times
regardless of demand. Omit to leave unchanged
--max-machines INTEGER Maximum number of machines of this machine type. Omit to leave
unchanged
--idle-timeout INTEGER Grace period to wait before terminating idle machines
(minutes). Omit to leave unchanged
--from-file FILENAME A path to a YAML or JSON file to load the arguments and
options from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell cluster update
Usage: spell cluster update [OPTIONS]
This command makes sure your Spell cluster is fully up to date and able to support
the latest features
Options:
-p, --profile TEXT AWS profile to pull credentials from
--help Show this message and exit.
spell cp
Usage: spell cp [OPTIONS] SOURCE_PATH [LOCAL_DIR]
Copy a file or directory from a finished run, uploaded resource, or public dataset
specified by SOURCE_PATH to a LOCAL_DIR.
The contents of SOURCE_PATH will be downloaded from Spell and written to LOCAL_DIR.
If LOCAL_DIR is not provided the current working directory will be used as a
default. If SOURCE_PATH is a directory, the contents of the directory will be
written to LOCAL_DIR.
Options:
-f, --force Overwrite all duplicate files.
-i, --ignore Ignore all duplicate files.
--help Show this message and exit.
spell feedback
Usage: spell feedback [OPTIONS] [MESSAGE]...
Submit feedback to the Spell team to identify bugs and improve Spell's products.
We at Spell would love to hear from you! Both positive and negative feedback is
appreciated. If reporting a bug, please provide as much detail as possible such as
git repo, framework, runid, and any steps to reproduce. If reporting a feature
request, we'll let you know as soon as it's been added.
Options:
--help Show this message and exit.
spell help
Usage: spell help [OPTIONS]
Display help information
Options:
--help Show this message and exit.
spell hyper
Usage: spell hyper [OPTIONS] COMMAND [ARGS]...
Create hyperparameter searches on Spell
Options:
--help Show this message and exit.
Commands:
bayesian Execute a hyperparameter bayesian search
grid Execute a hyperparameter grid search
kill Kill a hyperparameter search
random Execute a hyperparameter random search
stop Stop a hyperparameter search
spell hyper bayesian
Usage: spell hyper bayesian [OPTIONS] COMMAND [ARGS]...
Execute a hyperparameter bayesian search for COMMAND remotely on Spell's
infrastructure
The bayesian command is used to create parallelized bayesian optimization
hyperparameter optimization with num_runs number of total runs, parallelized in sets
of parallel_runs
Options:
-n, --num-runs INTEGER Maximum number of runs for the hyperparameter search
[required]
-r, --parallel-runs INTEGER Number of parallel runs to use for each iteration
[required]
--metric TEXT Metric name that will be used [required]
-a, --metric-agg [avg|min|max|last]
[required]
-p, --param NAME=MIN:MAX[:int|float]
Specify a hyperparameter for the run in the form: a
range (MIN:MAX) If unspecified, type float is assumed.
NAME should appear in the COMMAND surrounded by colons
(i.e., ":NAME:" to indicate where the VALUEs should be
substituted when creating each run.
--label TEXT Label to add to the run. Can specify more than one.
--tensorboard-dir PATH The path where tensorboard files will be read from
-f, --force Skip interactive prompts
-v, --verbose Print additional information
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-t, --machine-type [CPU|K80|V100]
Machine type to run on
--framework TEXT Machine learning framework to use. For a full list of
available frameworks see
https://spell.ml/docs/run_overview#dependency-
management-and-customization
-m, --mount RESOURCE[:MOUNT_PATH]
Attach a resource file or directory (e.g., from a run
output, public dataset, or upload). The resource
(specified by a Spell resource path) is required. An
optional mount path within the container can also be
specified, separated by a colon from the resource
name. If the mount path is omitted, it defaults to the
base name of the resource (e.g., 'mnist' for
'public/image/mnist'). Example: --mount
runs/42:/mnt/data
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell hyper grid
Usage: spell hyper grid [OPTIONS] COMMAND [ARGS]...
Execute a hyperparameter grid search for COMMAND remotely on Spell's infrastructure
The grid command is used to create numerous runs simultaneously to perform a
hyperparameter grid search. A run will be created for all possible combinations of
parameters provided with the --param option. All other options are the same as the
spell run command and will apply to every run created in the hyperparameter search.
Options:
--param NAME=VALUE[,VALUE,VALUE,...]
Specify a hyperparameter for the run. A run will be
created for all hyperparameter value combinations
provided. NAME should appear in the COMMAND surrounded
by colons (i.e., ":NAME:" to indicate where the VALUE
values should be substituted when creating each run.
--json-param NAME='[VALUE,VALUE,...]'
Specify a hyperparameter for the run. This should be
formatted as a JSON array. This can be used instead of
--param if you want a list of values which contain
commas. A run will be created for all hyperparameter
value combinations provided. NAME should appear in the
COMMAND surrounded by colons (i.e., ":NAME:" to
indicate where the VALUE values should be substituted
when creating each run.
--label TEXT Label to add to the run. Can specify more than one.
--tensorboard-dir PATH The path where tensorboard files will be read from
-f, --force Skip interactive prompts
-v, --verbose Print additional information
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-t, --machine-type [CPU|K80|V100]
Machine type to run on
--framework TEXT Machine learning framework to use. For a full list of
available frameworks see
https://spell.ml/docs/run_overview#dependency-
management-and-customization
-m, --mount RESOURCE[:MOUNT_PATH]
Attach a resource file or directory (e.g., from a run
output, public dataset, or upload). The resource
(specified by a Spell resource path) is required. An
optional mount path within the container can also be
specified, separated by a colon from the resource
name. If the mount path is omitted, it defaults to the
base name of the resource (e.g., 'mnist' for
'public/image/mnist'). Example: --mount
runs/42:/mnt/data
--stop-condition METRIC_NAME OPERATOR VALUE[:MIN_INDEX]
METRIC_NAME is the name of a metric the run produces.
OPERATOR is <, >, <=, or >=. VALUE is a float. During
the run if there is a metric value that meets the
condition the run will be stopped. You can optionally
provide a MIN_INDEX integer. In that case, only values
of the metric starting with that index will be
considered.
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell hyper kill
Usage: spell hyper kill [OPTIONS] HYPER-SEARCH-ID
Kill a hyperparameter search specified by HYPER-SEARCH-ID.
All runs in the hyperparameter search are killed. Any runs that are already in a
final state are unaffected.
Options:
-q, --quiet Suppress logging
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell hyper random
Usage: spell hyper random [OPTIONS] COMMAND [ARGS]...
Execute a hyperparameter random search for COMMAND remotely on Spell's
infrastructure
The random command is used to create numerous runs simultaneously to perform a
hyperparameter search. As many runs as specified with --num-runs will be created and
each hyperparameter specified with the --param option will be sampled to determine a
specific value for each run. All other options are the same as the spell run
command and will apply to every run created in the hyperparameter search.
Options:
-n, --num-runs INTEGER Total number of runs to create for the hyperparameter
search [required]
-p, --param NAME=VALUE[,VALUE,VALUE,...] | NAME=MIN:MAX[:linear|log|reverse_log[:int|float]]
Specify a hyperparameter for the run. Each run will
sample this random parameter specification to
determine a specific value for the run. The parameter
values can be provided as either a list of values
(from which one value will be randomly selected each
run) or a range (MIN:MAX) and optional scaling
('linear', 'log', or 'reverse_log') and type ('int' or
'float'). If unspecified, a linear scaling and type
float are assumed. NAME should appear in the COMMAND
surrounded by colons (i.e., ":NAME:" to indicate where
the VALUEs should be substituted when creating each
run.
--json-param NAME='[VALUE,VALUE,...]'
Specify a hyperparameter for the run. This should be
formatted as a JSON array. This can be used instead of
--param if you want a list of values which contain
commas. A run will be created for all hyperparameter
value combinations provided. NAME should appear in the
COMMAND surrounded by colons (i.e., ":NAME:" to
indicate where the VALUE values should be substituted
when creating each run.
--label TEXT Label to add to the run. Can specify more than one.
--tensorboard-dir PATH The path where tensorboard files will be read from
-f, --force Skip interactive prompts
-v, --verbose Print additional information
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-t, --machine-type [CPU|K80|V100]
Machine type to run on
--framework TEXT Machine learning framework to use. For a full list of
available frameworks see
https://spell.ml/docs/run_overview#dependency-
management-and-customization
-m, --mount RESOURCE[:MOUNT_PATH]
Attach a resource file or directory (e.g., from a run
output, public dataset, or upload). The resource
(specified by a Spell resource path) is required. An
optional mount path within the container can also be
specified, separated by a colon from the resource
name. If the mount path is omitted, it defaults to the
base name of the resource (e.g., 'mnist' for
'public/image/mnist'). Example: --mount
runs/42:/mnt/data
--stop-condition METRIC_NAME OPERATOR VALUE[:MIN_INDEX]
METRIC_NAME is the name of a metric the run produces.
OPERATOR is <, >, <=, or >=. VALUE is a float. During
the run if there is a metric value that meets the
condition the run will be stopped. You can optionally
provide a MIN_INDEX integer. In that case, only values
of the metric starting with that index will be
considered.
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell hyper stop
Usage: spell hyper stop [OPTIONS] HYPER-SEARCH-ID
Stop a hyperparameter search specified by HYPER-SEARCH-ID.
All runs in the hyperparameter search that are running are sent a stop signal that
ends current execution and transitions them to the "Saving" state. Stopped runs will
continue to transition through the "Pushing" and "Saving" steps after stopping. If
runs have not started yet, they are killed. Any runs that are already in a final
state are unaffected.
Options:
-q, --quiet Suppress logging
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell info
Usage: spell info [OPTIONS] RUN_ID
Displays information about a run including the start and end time as well run
parameters such as apts, pips, and mounts.
Options:
--help Show this message and exit.
spell jupyter
Usage: spell jupyter [OPTIONS] NAME
Create a Jupyter workspace on Spell.
The jupyter command is used to create a Jupyter workspace. If executed from within a
git respository, the HEAD commit (or optionally a different commit specified with
--commit-ref) will be copied into the Jupyter workspace. Alternatively, the
--github-url option can be used to specify a GitHub repository to copy into the
workspace. The other options can be used to further configure the Jupyter workspace
environment. Once created, the local web browser will open to the Jupyter workspace
in the Spell web console.
Options:
--lab Use Jupyter Lab (default: Jupyter Notebook)
--idle-kernel-timeout INTEGER Timeout in seconds after which to stop idle kernels
--no-activity-timeout INTEGER Timeout in seconds after which to stop the Jupyter
workspace if there are no running kernels and no user
activity
-t, --machine-type [CPU|K80|V100]
Machine type to run on
--framework TEXT Machine learning framework to use. For a full list of
available frameworks see
https://spell.ml/docs/run_overview#dependency-
management-and-customization
-m, --mount RESOURCE[:MOUNT_PATH]
Attach a resource file or directory (e.g., from a run
output, public dataset, or upload). The resource
(specified by a Spell resource path) is required. An
optional mount path within the container can also be
specified, separated by a colon from the resource
name. If the mount path is omitted, it defaults to the
base name of the resource (e.g., 'mnist' for
'public/image/mnist'). Example: --mount
runs/42:/mnt/data
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
-f, --force Skip interactive prompts
-v, --verbose Print additional information
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell keys
Usage: spell keys [OPTIONS] COMMAND [ARGS]...
Manage public SSH keys registered with Spell
With no subcommand, displays all of your public SSH keys
Options:
--help Show this message and exit.
Commands:
add Add a public SSH key to your account
generate Generate SSH key pair for your account
rm Remove a public SSH key from your account
spell keys add
Usage: spell keys add [OPTIONS]
Add a public SSH key to your account
Options:
-f, --file FILE location of public key file
-t, --title TEXT title for the public key
--help Show this message and exit.
spell keys generate
Usage: spell keys generate [OPTIONS]
Generate SSH key pair for your account
Options:
-t, --title TEXT title for the public key
-f, --force overwrite local key file if it already exists
--help Show this message and exit.
spell keys rm
Usage: spell keys rm [OPTIONS] [TITLE]
Remove a public SSH key from your account
Options:
--help Show this message and exit.
spell kill
Usage: spell kill [OPTIONS] RUN_IDS...
Kill a current run specified by RUN_ID. To batch kill, input multiple RUN_IDs
separated by spaces.
A killed run is sent a kill signal that ends current execution and immediately
transitions to the "Killed" state once the signal has been received. Killed runs do
not execute any steps after being killed, so a killed run will not, for example,
execute the "Pushing" or "Saving" steps if killed when in the "Running" status.
Options:
-q, --quiet Suppress logging
--help Show this message and exit.
spell label
Usage: spell label [OPTIONS] RUN_ID LABEL_NAME
Used to add and remove labels from runs
Options:
--remove Remove input label from run
--help Show this message and exit.
spell link
Usage: spell link [OPTIONS] [ALIAS] [RESOURCE_PATH]
With no argument, displays all existing symlinks.
With a single argument, searches through server database for a symlink with a
specific alias ALIAS.
With two arguments, creates a symlink with alias ALIAS to a finished run uploaded
resource, or public dataset with path RESOURCE_PATH. Note: ALIAS must NOT contain
any of the following: / . " \ [ ] : ; | = ,
Options:
--help Show this message and exit.
spell login
Usage: spell login [OPTIONS]
Log in with your username or email.
Prompts for the user's credentials and logs in to Spell. Both username and email are
valid for login. If you don't have an account with Spell, please create one at
https://spell.ml.
Options:
--identity TEXT your registered spell username or email
--password TEXT your registered spell password
--help Show this message and exit.
spell logout
Usage: spell logout [OPTIONS]
Log out of current session
Options:
--help Show this message and exit.
spell logs
Usage: spell logs [OPTIONS] RUN-ID
Retrieve logs for a run specified by RUN_ID.
Streams logs for the specified run. For runs with a large number of log lines the
`--tail N` option allows the user to print only the last N lines. When following
with `--follow` use `Ctrl + C` to detach.
Options:
-d, --delay Replay delay between log entries
-f, --follow Follow log output
-n, --tail INTEGER Show the last NUM lines
-v, --verbose Print additional information
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell ls
Usage: spell ls [OPTIONS] [PATH]
List resource files for datasets, run outputs, and uploads.
Resources are the generic name for datasets, models, or any other files that can be
made available to a run. Spell keeps these organized for you in a remote filesystem
that is browsable with the `ls` command, with the resources placed into directories
that reflect their origin.
There are many ways resources are generated. The user can upload resources directly
with `spell upload` or execute a run with `spell run` that writes files to disk.
Spell also provides a number of publicly-accessible datasets.
Options:
-h Display file sizes in human-readable format
--help Show this message and exit.
spell model-servers
Usage: spell model-servers [OPTIONS] COMMAND [ARGS]...
Manage model servers
With no subcommand, displays all of your model servers
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
Commands:
apply Update model servers based on a user-specified YAML file
create Create a new model server
info Get info about a model server
logs Get logs from a model server
renew-token Renews the access token for model server
rm Remove a model server
start Start a model server
status Get the status of all pods for this server.
stop Stop a model server
update Update a model server with new configuration
spell model-servers apply
Usage: spell model-servers apply [OPTIONS] PATH
Create/update/delete multiple model servers with a YAML file
Properties:
specifier REQUIRED (ex: image_classifier:v1)
path REQUIRED (ex: runs/8/mymodel.pt)
type OPTIONAL (ex: tensorflow OR pytorch. Use auto-detected type if not specified)
owner OPTIONAL (Uses this value if specified, if not then cmd line argument is used if specified, and finally current active spell owner is used as a default if neither is specified)
clusterName OPTIONAL (Uses this value if specified, if not then cmd line argument is used if specified. If neither is specified, the spell owner should have exactly ONE cluster, which will be used as default)
Options:
-d, --delete Delete any currently existing model servers in this cluster that
are missing from the YAML file
--owner TEXT Default owner for model servers unspecified in the YAML file.
Only use if you would like to specify an alternative owner to
the currently active one.
--cluster-name TEXT Default cluster name for model servers unspecified in the YAML
file
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers create
Usage: spell model-servers create [OPTIONS] NAME PATH
Create a new Tensorflow model server with the specified NAME from resources stored
at PATH. NAME should be fully qualified, following the format: <model_name>:<tag>
Options:
-t, --type [tensorflow|pytorch]
Please see the documentation located at
https://spell.ml/docs/model_servers
to correctly
structure your model for the various supported types.
--cluster-name TEXT Name of the cluster where the model server will be
created
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell model-servers info
Usage: spell model-servers info [OPTIONS] NAME
Get info about the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers logs
Usage: spell model-servers logs [OPTIONS] NAME
Get logs for the model server with the specified NAME
Options:
-f, --follow Follow log output
-p, --pod TEXT The ID of the pod you would like logs for. Omit to get a list of
all pods.
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers renew-token
Usage: spell model-servers renew-token [OPTIONS] NAME
Renews the access token for model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers rm
Usage: spell model-servers rm [OPTIONS] NAME
Remove the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers start
Usage: spell model-servers start [OPTIONS] NAME
Start the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers status
Usage: spell model-servers status [OPTIONS] NAME
Get the status of all pods for this server.
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers stop
Usage: spell model-servers stop [OPTIONS] NAME
Stop the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell model-servers update
Usage: spell model-servers update [OPTIONS] NAME PATH
Update the model server with the specified NAME to have a new configuration
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell passwd
Usage: spell passwd [OPTIONS]
Set a new password for your Spell account.
Prompts for the user's current password before allowing the user to specify a new
password.
Options:
--help Show this message and exit.
spell ps
Usage: spell ps [OPTIONS]
Display all user runs and their details
Displays information about all of the user's active and historical runs, displayed
oldest to newest. It can be used to keep track of current active runs or find a
historical run.
Options:
-n, --number INTEGER The number of runs to display
--trunc / --no-trunc Truncate command output
-r, --repository TEXT Repository name to filter runs by
-l, --label TEXT Label name to filter runs by. Multiple usages will return runs
with ANY of the input labels
-v, --verbose Include extra columns such as git commit hash and description
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell repos
Usage: spell repos [OPTIONS]
List all owner repositories
A repository is defined by the root commit of a git repository. Thus, the family of
all Git commits that originate from the same root commit belong to the same
repository. Repositories do not need to be managed or created -- they are created by
the run command when necessary.
Options:
--help Show this message and exit.
spell rm
Usage: spell rm [OPTIONS] RESOURCES...
Remove one or more uploaded resources.
The removed resources will no longer show up in `ls` or be mountable with `--mount`.
Options:
--help Show this message and exit.
spell run
Usage: spell run [OPTIONS] COMMAND [ARGS]...
Execute COMMAND remotely on Spell's infrastructure
The run command is used to create runs and is likely the command you'll use most
while using Spell. It is intended to be emulate local development. Any code,
software, binaries, etc., that you can run locally on your computer can be run on
Spell - you simply put `spell run` in front of the same commands you would use
locally and they will run remotely. The various options can be used to customize the
environment in which COMMAND will run.
Options:
-t, --machine-type [CPU|K80|V100]
Machine type to run on
--framework TEXT Machine learning framework to use. For a full list of
available frameworks see
https://spell.ml/docs/run_overview#dependency-
management-and-customization
-m, --mount RESOURCE[:MOUNT_PATH]
Attach a resource file or directory (e.g., from a run
output, public dataset, or upload). The resource
(specified by a Spell resource path) is required. An
optional mount path within the container can also be
specified, separated by a colon from the resource
name. If the mount path is omitted, it defaults to the
base name of the resource (e.g., 'mnist' for
'public/image/mnist'). Example: --mount
runs/42:/mnt/data
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
--tensorboard-dir PATH The path where tensorboard files will be read from
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
--label TEXT Label to add to the run. Can specify more than one.
-f, --force Skip interactive prompts
-v, --verbose Print additional information
-b, --background Do not print logs
--distributed N Execute a distributed run using N machines of the
specified machine type.
--stop-condition METRIC_NAME OPERATOR VALUE[:MIN_INDEX]
METRIC_NAME is the name of a metric the run produces.
OPERATOR is <, >, <=, or >=. VALUE is a float. During
the run if there is a metric value that meets the
condition the run will be stopped. You can optionally
provide a MIN_INDEX integer. In that case, only values
of the metric starting with that index will be
considered.
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.
spell server
Usage: spell server [OPTIONS] COMMAND [ARGS]...
Manage model servers
With no subcommand, displays all of your model servers
Options:
--help Show this message and exit.
Commands:
info Get info about a model server
logs Get logs from a model server
rm Remove a model server
start Start a model server
stop Stop a model server
update Update a model server with new configuration
spell server info
Usage: spell server info [OPTIONS] NAME
Get info about the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell server logs
Usage: spell server logs [OPTIONS] NAME
Get logs for the model server with the specified NAME
Options:
-f, --follow Follow log output
-p, --pod TEXT The ID of the pod you would like logs for. Omit to get a list of
all pods.
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell server rm
Usage: spell server rm [OPTIONS] NAME
Remove the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell server start
Usage: spell server start [OPTIONS] NAME
Start the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell server stop
Usage: spell server stop [OPTIONS] NAME
Stop the model server with the specified NAME
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell server update
Usage: spell server update [OPTIONS] NAME PATH
Update the model server with the specified NAME to have a new configuration
Options:
--from-file FILENAME A path to a YAML or JSON file to load the arguments and options
from
--save-command FILE Save the arguments and options to this command to a file
--help Show this message and exit.
spell stats
Usage: spell stats [OPTIONS] RUN_ID
Display performance statistics for a run specified by RUN_ID
A run must have status 'Running' to display performance statistics. CPU statistics
(e.g., memory usage, CPU utilization) and, if applicable for the machine type, GPU
statistics (e.g., power and GPU memory utilization) are displayed.
Options:
-f, --follow Continually stream stats for the run
--help Show this message and exit.
spell status
Usage: spell status [OPTIONS]
Display account and billing information
Display account and billing information, such as current plan, total run time, and
free credit usage.
Options:
--help Show this message and exit.
spell stop
Usage: spell stop [OPTIONS] RUN_IDS...
Stop runs currently in the "Running" state specified by RUN_IDS.
A stopped run is sent a stop signal that ends current execution and transitions to
the "Saving" state once the signal has been received. Stopped runs will execute any
and all steps after being stopped, so a stopped run will, for example, execute both
the "Pushing" and "Saving" steps after stopping. If "stop" is called on a run that
has not yet entered the "Running" state, the run is killed (see documentation of
"kill" for more details).
Options:
-q, --quiet Suppress logging
--help Show this message and exit.
spell unlink
Usage: spell unlink [OPTIONS] ALIAS
Remove an existing symlink by referencing a particular link alias.
Options:
--help Show this message and exit.
spell upload
Usage: spell upload [OPTIONS] PATH
Upload the contents of PATH to uploads/NAME
The contents of PATH will be uploaded to Spell and will be accessible at
uploads/NAME. If NAME is not provided and path is a directory, NAME defaults to the
the directory name of PATH. If NAME is not provided and PATH is a file, the user is
prompted for a NAME.
Options:
-n, --name NAME upload name
--cluster-name TEXT cluster name
-c, --compress compress before uploading
--help Show this message and exit.
spell whoami
Usage: spell whoami [OPTIONS]
Display current user information.
Display information about the currently logged in user, such as username, email, and
various other metadata.
Options:
--help Show this message and exit.
spell workflow
Usage: spell workflow [OPTIONS] COMMAND [ARGS]...
Execute WORKFLOW either remotely or locally
The workflow command is used to create workflows which manage other runs. Complex
machine learning applications often require multi-stage pipelines (e.g., data
loading, transforming, training, testing, iterating). Workflows are designed to help
you automate this process. While a workflow executes much like a normal run, it is
capable of launching other runs that are all associated with each other. A workflow
must specify every git commit that will be used by the given workflow script using
the `--repo` flag. The various other options can be used to customize the
environment that the workflow script runs in.
Options:
--local Execute command locally instead of remotely on Spell's
infrastructure
-r, --repo LABEL=PATH[:COMMIT_HASH]
Add a git repository at a specific commit to this
workflow by specifying LABEL=PATH, where PATH is a
path to a local git repository on disk and LABEL is a
label to refer to this snapshot in future run
requests. COMMIT_HASH can optionally be specified by
LABEL=PATH[:COMMIT_HASH]. If no COMMIT_HASH is
specified, the currently checked out commit of the
repo will be used.
--github-repo LABEL=URL[:REF] Add a GitHub repository at a specific ref to this
workflow by specifying LABEL=URL, where URL is a url
to a GitHub repository and LABEL is a label to refer
to this snapshot in future run requests. REF can
optionally be specified by LABEL=PATH[:REF], it can be
a branch, commit hash, or label, and will be resolved
to a commit hash immediately. If no REF is specified,
master will be used.
--conda-file FILE Path to conda specification file or YAML environment
file
--docker-image, --docker_image, --from TEXT
Custom docker image to run from. Specify image as
<domain>/<repository>/<image_name>:<tag>. Default to
docker.io if <domain>/ is omitted. To use images from
Google Container Registry, first run 'spell cluster
add-docker-registry' as a gcp cluster owner.
--pip TEXT Single dependency to install using pip
--pip-req TEXT Requirements file to install using pip
--apt TEXT Dependency to install using apt
-e, --env TEXT Add an environment variable to the run
-c, --commit-ref TEXT Git commit hash to run
--github-url TEXT GitHub URL of a repository to run
--github-ref TEXT commit hash, branch, or tag of the repository to pull
(default: 'master')
-d, --description TEXT Description of the run. If unspecified defaults to the
current commit message
-f, --force Skip interactive prompts
-v, --verbose Print additional information
-b, --background Do not print logs
--from-file FILENAME A path to a YAML or JSON file to load the arguments
and options from
--save-command FILE Save the arguments and options to this command to a
file
--help Show this message and exit.