mattertune.configs.finetune.lr_scheduler

class mattertune.configs.finetune.lr_scheduler.CosineAnnealingLRConfig(*, type='CosineAnnealingLR', T_max, eta_min=0, last_epoch=-1)

Parameters:

• type (Literal['CosineAnnealingLR'])

• T_max (int)

• eta_min (float)

• last_epoch (int)

type: Literal['CosineAnnealingLR']

Type of the learning rate scheduler.

T_max: int

Maximum number of iterations.

eta_min: float

Minimum learning rate.

last_epoch: int

The index of last epoch.

class mattertune.configs.finetune.lr_scheduler.ExponentialConfig(*, type='ExponentialLR', gamma)

Parameters:

• type (Literal['ExponentialLR'])

• gamma (float)

type: Literal['ExponentialLR']

Type of the learning rate scheduler.

gamma: float

Multiplicative factor of learning rate decay.

class mattertune.configs.finetune.lr_scheduler.MultiStepLRConfig(*, type='MultiStepLR', milestones, gamma)

Parameters:

• type (Literal['MultiStepLR'])

• milestones (list[int])

• gamma (float)

type: Literal['MultiStepLR']

Type of the learning rate scheduler.

milestones: list[int]

List of epoch indices. Must be increasing.

gamma: float

Multiplicative factor of learning rate decay.

class mattertune.configs.finetune.lr_scheduler.ReduceOnPlateauConfig(*, type='ReduceLROnPlateau', mode, factor, patience, threshold=0.0001, threshold_mode='rel', cooldown=0, min_lr=0, eps=1e-08)

Parameters:

• type (Literal['ReduceLROnPlateau'])

• mode (str)

• factor (float)

• patience (int)

• threshold (float)

• threshold_mode (str)

• cooldown (int)

• min_lr (float)

• eps (float)

type: Literal['ReduceLROnPlateau']

Type of the learning rate scheduler.

mode: str

One of {“min”, “max”}. Determines when to reduce the learning rate.

factor: float

Factor by which the learning rate will be reduced.

patience: int

Number of epochs with no improvement after which learning rate will be reduced.

threshold: float

Threshold for measuring the new optimum.

threshold_mode: str

One of {“rel”, “abs”}. Determines the threshold mode.

cooldown: int

Number of epochs to wait before resuming normal operation.

min_lr: float

A lower bound on the learning rate.

eps: float

Threshold for testing the new optimum.

class mattertune.configs.finetune.lr_scheduler.StepLRConfig(*, type='StepLR', step_size, gamma)

Parameters:

• type (Literal['StepLR'])

• step_size (int)

• gamma (float)

type: Literal['StepLR']

Type of the learning rate scheduler.

step_size: int

Period of learning rate decay.

gamma: float

Multiplicative factor of learning rate decay.