Gradients

Collection of gradient Ivy functions.

ivy.adam_update(ws, dcdws, lr, mw, vw, step, beta1=0.9, beta2=0.999, epsilon=1e-07, f=None)[source]

Update weights ws of some function, given the derivatives of some cost c with respect to ws, using ADAM update. [reference]

Parameters
  • ws (container of variables) – Weights of the function to be updated.

  • dcdws (container of arrays) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].

  • lr (float) – Learning rate, the rate at which the weights should be updated relative to the gradient.

  • mw (container of arrays) – running average of the gradients

  • vw (container of arrays) – running average of second moments of the gradients

  • step (int) – training step

  • beta1 (float) – gradient forgetting factor

  • beta2 (float) – second moment of gradient forgetting factor

  • epsilon (float) – divisor during adam update, preventing division by zero

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

The new function weights ws_new, and also new mw and vw, following the gradient descent updates.

ivy.execute_with_gradients(func, xs, f=None)[source]

Call function func with input of xs variables, and return func first output y, the gradients [dy/dx for x in xs], and any other function outputs after the returned y value

Parameters
  • func (function) – Function for which we compute the gradients of the output with respect to xs input.

  • xs (sequence of variables) – Variables for which to compute the function gradients with respective to.

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

the function first output y, the gradients [dy/dx for x in xs], and any other extra function outputs

ivy.gradient_descent_update(ws, dcdws, lr, f=None)[source]

Update weights ws of some function, given the derivatives of some cost c with respect to ws, [dc/dw for w in ws].

Parameters
  • ws (Ivy container) – Weights of the function to be updated.

  • dcdws (Ivy container) – Derivates of the cost c with respect to the weights ws, [dc/dw for w in ws].

  • lr (float) – Learning rate, the rate at which the weights should be updated relative to the gradient.

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

The new function weights ws_new, following the gradient descent updates.

ivy.is_variable(x, f=None)[source]

Determines whether the input is a variable or not.

Parameters
  • x (array) – An ivy array.

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

Boolean, true if x is a trainable variable, false otherwise.

ivy.stop_gradient(x, f=None)[source]

Stops gradient computation.

Parameters
  • x (array) – Array for which to stop the gradient.

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

The same array x, but with no gradient information.

ivy.variable(x, f=None)[source]

Creates a variable, which supports gradient computation.

Parameters
  • x (array) – An ivy array.

  • f (ml_framework, optional) – Machine learning framework. Inferred from inputs if None.

Returns

An ivy variable, supporting gradient computation.