Optimizer Schedules

constant_schedule(value)	Constructs a constant schedule.
cosine_decay_schedule(init_value, decay_steps)	Returns a function which implements cosine learning rate decay.
cosine_onecycle_schedule(transition_steps, ...)	Returns a function which implements the onecycle learning rate schedule.
exponential_decay(init_value, ...[, ...])	Constructs a schedule with either continuous or discrete exponential decay.
join_schedules(schedules, boundaries)	Sequentially apply multiple schedules.
linear_onecycle_schedule(transition_steps, ...)	Returns a learning rate with three linear phases.
linear_schedule(init_value, end_value, ...)	Schedule with linear transition from init_value to end_value.
piecewise_constant_schedule(init_value[, ...])	Piecewise constant schedule with scaled jumps at specific boundaries.
piecewise_interpolate_schedule(...[, ...])	Piecewise interpolated schedule with linear or cosine transitions.
polynomial_schedule(init_value, end_value, ...)	Constructs a schedule with polynomial transition from init to end value.
sgdr_schedule(cosine_kwargs)	SGD with warm restarts.
warmup_constant_schedule(init_value, ...)	Linear warmup followed by constant schedule i.e no decay.
warmup_cosine_decay_schedule(init_value, ...)	Linear warmup followed by cosine decay.
warmup_exponential_decay_schedule(...[, ...])	Linear warmup followed by exponential decay.
Schedule
InjectHyperparamsState(count, hyperparams, ...)	Deprecated class kept for backwards compatibility.
inject_hyperparams(inner_factory[, ...])	Wrapper to injects stateful hyperparameters into GradientTransformations.

Reduce on plateau

reduce_on_plateau([factor, patience, rtol, ...])

Reduce learning rate when a metric has stopped improving.