Optimizers

Direct Compression Optimizer

class condensa.opt.DC

Condensa direct compression optimizer.

__init__()

Initialize self. See help(type(self)) for accurate signature.

compress(w, pi, delta, trainloader, testloader, valloader, criterion)

Performs model compression using direct optimization.

Parameters

  • w (torch.nn.Module) – PyTorch model.

  • pi – Compression function.

  • delta – Decompression function.

  • trainloader – Training dataloader.

  • testloader – Test dataloader.

  • valloader – Validation dataloader.

  • criterion – Loss criterion.

L-C Optimizer

class condensa.opt.LC(steps=30, l_optimizer=None, l_optimizer_params={}, lr=None, lr_end=None, lr_decay=None, lr_schedule=None, lr_multiplier=None, mb_iterations_per_l=0, mb_iterations_first_l=0, mu_init=0.0, mu_multiplier=1.0, mu_cap=10000, distributed=False, debugging_flags={})

Condensa L-C compression engine.

__init__(steps=30, l_optimizer=None, l_optimizer_params={}, lr=None, lr_end=None, lr_decay=None, lr_schedule=None, lr_multiplier=None, mb_iterations_per_l=0, mb_iterations_first_l=0, mu_init=0.0, mu_multiplier=1.0, mu_cap=10000, distributed=False, debugging_flags={})

Constructs an LC class instance.

Parameters

  • steps (float) – Number of L-C iterations.

  • l_optimizer – L-step optimizer to use.

  • l_optimizer_params (dict) – L-step optimizer hyper-parameters.

  • lr (float) – Starting learning rate.

  • lr_end (float) – Ending learning rate.

  • lr_schedule (List) – Learning rate schedule.

  • lr_multiplier (float) – Learning rate multiplier.

  • mb_iterations_per_l (int) – Number of mini-batch iterations per L-step.

  • mb_iterations_first_l (int) – Number of mini-batch iterations for first L-step.

  • mu_init (float) – Initial value of mu.

  • mu_multiplier (float) – Mu multiplier.

  • mu_cap (float) – Maximum permitted value for mu.

  • distributed (bool) – Enable/disable data-parallelism in L-step.

  • debugging_flags (dict) – Debugging flags

compress(w, pi, delta, trainloader, testloader, valloader, loss_fn)

Main L-C compression method.

Parameters

  • w (torch.nn.Module) – Input model.

  • pi – Compression function.

  • delta – Decompression function.

  • trainloader – Training dataloader.

  • testloader – Test dataloader.

  • valloader – Validation dataloader.

  • loss_fn – Loss criterion.

zero_(model)

Zeroes out model parameters.

Parameters

model (torch.nn.Module) – PyTorch model.