A Guide to the Complex-Step Derivative Approximation
« BackThe complex-step is a method for estimating derivatives. This is a guide to the theory and implementation of the complex-step derivative approximation.
Introduction
The complex-step derivative approximation is a very convenient way of estimating derivatives numerically. It is a simple and accurate means of finding derivatives of a quantity calculated by an existing algorithm.
If f is the result of any (real-valued) numerical algorithm, and x is one of the input variables, then the derivative ∂f/∂x is simply
\[\frac{\partial f}{\partial x} \approx \frac{Im\left[ f(x+ih) \right]}{h}\]where Im is the imaginary part, i is the imaginary unit, and h is a small step size. How small? h = 1.0E-200 works very well in nearly all cases, and results in derivatives no less accurate than the original algorithm. In some instances, different architectures may give noticeable errors at smaller step sizes so if you are unsure and it is possible, check the computed derivatives.
The complex-step derivative approximation is a second order accurate formula analogous to finite-differencing, but it does not have a subtraction operation and therefore it is not subject to subtractive cancellation, which we demonstrate in the plot below. Further details are available in the materials referenced below and in the complex-step paper.
Implementation
The implementation details of the complex-step derivative approximation are different for each programming language.
A noteworthy alternative to the complex-step method is the dual numbers approach. Please see this page for the implementation details.