Optimizer

class ivy.neural_net_stateful.optimizers.Optimizer(lr, dev_str, compile_step=False)[source]

Bases: abc.ABC

__init__(lr, dev_str, compile_step=False)[source]

Construct an general Optimizer. This is an abstract class, and must be derived.

Parameters
  • lr (function or float.) – Learning rate.

  • compile_step (bool, option) – Whether to compile the optimizer step, default is False.

abstract set_state(state)[source]

Set state of the optimizer.

Parameters

state (Ivy container of state tensors) – Nested state to update.

step(v, grads)[source]

Update nested variables container v from possibly compiled overriden private self._step_fn

Parameters
  • v (Ivy container of variables) – Nested variables to update.

  • grads (sequence of arrays) – Nested gradients to update.

Returns

The updated variables, following update step.


Supported Frameworks:

empty jax_logo empty tf_logo empty pytorch_logo empty mxnet_logo empty numpy_logo empty