Integrating with Weights & Biases (W&B)

Weights & Biases, which provides insight tools for machine learning, seamlessly integrates with Spell. This integration allows for much deeper insights into training metrics and examples of a model. It enables Spell runs to send metrics directly to your Weights & Biases account.

The steps in this guide demonstrate how to enable the W&B integration for your account. Every run, whether it is within a Developer or Teams account, may utilize this integration.

Adding the Weights & Biases Integration

Step 1. Navigate to your integration settings page.

Click the "Install" button in the Weights & Biases section.

Image of integration settings page

Step 2. Navigate to the W&B user settings page. Under API Tokens use an existing token or create a new token. Copy your token.

Image of W&B settings page indicating Api Key location

Step 3. Paste your token. Click the "Confirm" button.

Image of integration settings page showing W&B modal

Step 4. View the integration listed in your integrations page. You may remove it from the list at any time.

Running A Run

Create scripts that integrate the wandb library without running wandb login before your script. We suggest that you follow a simple PyTorch W&B example. This can be done directly by running the following command:

spell run --github-url https://github.com/wandb/examples.git --machine-type k80 'python pytorch-cnn-mnist/main.py'

Once W&B integration is active, you do not need to add wandb as a pip package or run wandb login. This process is now automated by Spell. The run starts as follows:

💫 Casting spell #1…
✨ Stop viewing logs with ^C
✨ Building… done
✨ Machine_Requested… done
✨ Run is running
wandb: Started W&B process version 0.8.10 with PID 17
wandb: Local directory: wandb/run-20190909_212858-46pckj1w
wandb: Syncing run silvery-snow-21: https://app.wandb.ai/user123/uncategorized/runs/46pckj1w
wandb: Run `wandb off` to turn off syncing.

Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz
Extracting ../data/MNIST/raw/train-images-idx3-ubyte.gz to ../data/MNIST/raw
Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ../data/MNIST/raw/train-labels-idx1-ubyte.gz
Extracting ../data/MNIST/raw/train-labels-idx1-ubyte.gz to ../data/MNIST/raw
Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw/t10k-images-idx3-ubyte.gz
Extracting ../data/MNIST/raw/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw
Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz
Extracting ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw
Processing...
Done!
Train Epoch: 1 [0/60000 (0%)]   Loss: 2.329947
Train Epoch: 1 [640/60000 (1%)] Loss: 2.305957
...

Navigate to your Runs page to view Run Details. The W&B link is listed towards the bottom of the Run Details section.

Image of run details page showing W&B button

Click on "Open" to view the corresponding run metrics in W&B:

Image of W&B run page