Skip to main content

Fastai

If you're using fastai to train your models, W&B has an easy integration using the WandbCallback. Explore the details in interactive docs with examples →

Log with W&B

a) Sign up for a free account at https://wandb.ai/site and then log in to your wandb account.

b) Install the wandb library on your machine in a Python 3 environment using pip

c) log in to the wandb library on your machine. You will find your API key here: https://wandb.ai/authorize.

pip install wandb
wandb login

Then add the WandbCallback to the learner or fit method:

import wandb
from fastai.callback.wandb import *

# start logging a wandb run
wandb.init(project="my_project")

# To log only during one training phase
learn.fit(..., cbs=WandbCallback())

# To log continuously for all training phases
learn = learner(..., cbs=WandbCallback())
info

If you use version 1 of Fastai, refer to the Fastai v1 docs.

WandbCallback Arguments

WandbCallback accepts the following arguments:

ArgsDescription
logWhether to log the model's: "gradients" , "parameters", "all" or None (default). Losses & metrics are always logged.
log_predswhether we want to log prediction samples (default to True).
log_preds_every_epochwhether to log predictions every epoch or at the end (default to False)
log_modelwhether we want to log our model (default to False). This also requires SaveModelCallback
model_nameThe name of the file to save, overrides SaveModelCallback
log_dataset
  • False (default)
  • True will log folder referenced by learn.dls.path.
  • a path can be defined explicitly to reference which folder to log.

Note: subfolder "models" is always ignored.

dataset_namename of logged dataset (default to folder name).
valid_dlDataLoaders containing items used for prediction samples (default to random items from learn.dls.valid.
n_predsnumber of logged predictions (default to 36).
seedused for defining random samples.

For custom workflows, you can manually log your datasets and models:

  • log_dataset(path, name=None, metadata={})
  • log_model(path, name=None, metadata={})

Note: any subfolder "models" will be ignored.

Distributed Training

fastai supports distributed training by using the context manager distrib_ctx. W&B supports this automatically and enables you to track your Multi-GPU experiments out of the box.

A minimal example is shown below:

import wandb
from fastai.vision.all import *
from fastai.distributed import *
from fastai.callback.wandb import WandbCallback

wandb.require(experiment="service")
path = rank0_first(lambda: untar_data(URLs.PETS) / "images")


def train():
dls = ImageDataLoaders.from_name_func(
path,
get_image_files(path),
valid_pct=0.2,
label_func=lambda x: x[0].isupper(),
item_tfms=Resize(224),
)
wandb.init("fastai_ddp", entity="capecape")
cb = WandbCallback()
learn = vision_learner(dls, resnet34, metrics=error_rate, cbs=cb).to_fp16()
with learn.distrib_ctx(sync_bn=False):
learn.fit(1)


if __name__ == "__main__":
train()

Then, in your terminal you will execute:

$ torchrun --nproc_per_node 2 train.py

in this case, the machine has 2 GPUs.

Logging only on the main process

In the examples above, wandb launches one run per process. At the end of the training, you will end up with two runs. This can sometimes be confusing, and you may want to log only on the main process. To do so, you will have to detect in which process you are manually and avoid creating runs (calling wandb.init in all other processes)

import wandb
from fastai.vision.all import *
from fastai.distributed import *
from fastai.callback.wandb import WandbCallback

wandb.require(experiment="service")
path = rank0_first(lambda: untar_data(URLs.PETS) / "images")


def train():
cb = []
dls = ImageDataLoaders.from_name_func(
path,
get_image_files(path),
valid_pct=0.2,
label_func=lambda x: x[0].isupper(),
item_tfms=Resize(224),
)
if rank_distrib() == 0:
run = wandb.init("fastai_ddp", entity="capecape")
cb = WandbCallback()
learn = vision_learner(dls, resnet34, metrics=error_rate, cbs=cb).to_fp16()
with learn.distrib_ctx(sync_bn=False):
learn.fit(1)


if __name__ == "__main__":
train()

in your terminal call:

$ torchrun --nproc_per_node 2 train.py

Examples

Was this page helpful?👍👎