Differential Matrix Calculus

The Derivative of a Matrix

Consider where and is a scalar. The derivative of with respect to time is:

The derivative of a matrix with respect to a scalar is just the derivative of all the values. Similarly for an matrix

Vector-Valued Functions

The set of functions on the same variables can be represented as a vector-valued function over the vector

Each element of the vector is a function of the variables

  • is an vector function over
  • is an vector

The Matrix Form of the Chain Rule

If and such that :

This is the same as the scalar case, but note that matrix multiplication is not commutative so the order matters.

The Jacobian Matrix

The derivative of a vector function with respect to a column vector is defined formally as the Jacobian matrix:

The Jacobian matrix is the derivative of a multivariate function, representing the best linear approximation to a differentiable function near a point. Geometrically, it defines a tangent plane to the function at the point

Linearisation of a Matrix Differential Equation

Assume that is a stationary point (equilibrium state) of a non-linear system described by a matrix differential equation:

The linearisation of this system is the evaluation of the Jacobian matrix at . The linearised equation is , with the matrix of constants .


Linearise the system around an equilibrium state:

, , and are parameters. At it's equilibrium,

There are three solutions to this system of algebraic equations, but we're interested in the one at the origin where . Evaluating the Jacobian at this point:

The linearised equation is therefore:

The Derivative of a Scalar Function With Respect to a Vector

If is a scalar quantity that depends on a vector of variables, then the derivative of with respect to \mathbf is a row vector:

This is the gradient or nabla ()

The Derivative of the Quadratic Form

Using an auxillary result

We can compute the derivative of a quadratic form :

Since is symmetric by definition of the quadratic form, , the derivative of the quadratic form is a row vector:


Consider the polynomial . Find . First putting the equation into quadratic form:

The derivative :