Better minimization procedures are based on the Newton-Raphson method (second-order).

For a continuous, differentiable function (in one dimension) f(x), at a minimum (x=x*)the first derivative is zero:

d(f(x*))/dx = f '(x*) = 0

Starting from position x (not the minimum),

x* = x + dx

Therefore

f ' (x + dx) = 0

This can be written as a Taylor series

f ' (x + dx) = f '(x) + f ''(x)dx + f '''(x)dx² + …

This series is truncated at the second term (equivalent to assuming the function is quadratic). and set equal to zero:

f '(x) + f ''(x)dx = 0

Rearranging gives the Newton-Raphson step,

dx = –f '(x)/f ''(x)

Therefore

x* = x – f '(x)/f ''(x)

Next: features of the Newton-Raphson method