In engineering, many problems require one to find the stationary point or root of a function f(x) = 0. High order functions, such as x¹⁰ + 7x⁹ + 4x⁷ + 8x⁴ + 9x³ + 70 = 0, are extremely difficult to solve by hand. However, numerical methods like Newton's method can find the approximate root for such equations.

The prerequisite of using Newton's method, also known as Newton-Raphson method, is that the function must be differentiable and point x₁, the initial estimate, must be close to a solution of the equation f(x) = 0. Numerical methods require numerous steps use the derivative of the function to "zero in " to the answer. This is done by first constructing function's tangent line L₁ from point (x₁, f(x₁)). Next, find the intersection of the x axis and L₁ which is (x₂, 0). It is helpful to remember that the equation of a line is

     y - y₁ = m(x-x₁)

where m is the slope. As the slope of L₁ is the derivative of the function and (x₁, f(x₁)) is a given point, the equation of L₁ can be expressed as

     y₂ - f(x₁) = f ^'(x₁)(x₂ - x₁)

Since L₁ intersects x axis, the value of y₂ equals 0. The equation can be simplified to

     - f(x₁) = f ^'(x₁)(x₂ - x₁)

When f ^'(x₁) ≠ 0, x₂ is obtained

     x₂ = x₁ - f(x₁)/f ^'(x₁)

Next, repeat this iteration from x₂. The intersection, x₃, of tangent line L₂ and x axis, is found.

     x₃ = x₂ - f(x₂)/f ^'(x₂)

By repeating this iteration, a series of points x₁, x₂, x₃, ....x_n is obtained. The value of x_n+1 is

     x_n+1 = x_n - f(x_n)/f ^'(x_n)

This value approaches to the stationary point of the function.

The concept of Newton's Method can be understand by finding the stationary point around (9, 75) for function y = x² - 6.

To compare with Newton's method later, the exact answer is easily determined. When y = 0,

     x² - 6 = 0

Therefore, x = 2.449 and x = -2.449, the stationary point around (9, 75) is x = 2.449. In the diagram it is s = 2.449.

Now use the Newton's method to find the stationary point.The slope of the tangent line for function y = x² - 6 is

     m = dy/dx = 2x

At x = 9 point, m = 2(9) = 18.

Recall that the equation for the tangent line is

     y - y₁ = m(x - x₁)

First iteration

Substitute (9, 75) and m = 18 into the line equation gives,

     y - 75 = 18(x-9)

When stationary point is the point when y = 0, thus,

     0-75 = 18(x-9)

so x = 4.833

That is x₂ = 4.833. The relative error = (x₂ - s)/s = (4.833 - 2.449)/2.449 = 97.3%. This error is too large to accept. Therefore more iterations are needed.

Second iteration

Find the function's value when x₂ = 4.833.

     f(4.833) = (4.833)² - 6 = 17.358

Substituting (4.833, 17.358) and m = 2x = 2(4.833) = 9.666 into y - y₂ = m(x - x₂) gives:

     y - 17.358 = 9.666(x - 4.833)

Setting y = 0 gives

     0 -17.358 = 9.666(x - 4.833)

So x = 3.037. That is x₃ = 3.037. The relative error = (x₃ - s)/s = (3.037 - 2.449)/2.449 = 24%

It is obvious that the relative error reduces and the value approaches real stationary point. If more iterations are done, the accuracy increases.

In the above example, Newton's method was able to find the root of equation x² - 6 = 0, but in some cases Newton's method can fail for various reasons.Some examples are listed below.

• There will be a problem for the function y = x² - 4x +15 if x = 2 is used as the initial point.
  The derivative of y is f ^'(x) = 2x - 4. At point x = 2, f ^'(x) = 2x - 4 = 2(2) - 4 = 0.
  Recall the formula of Newton's method is
  x_n+1 = x_n - f(x_n)/f ^'(x_n) which requires f ^'(x₁) ≠ 0. Therefore, Newton's method fails in this example.
  
• For some rapidly changing functions, such as y = x^1/3 problems can occur.
  The derivative of y is
  f ^'(x) = (x^-2/3)/3
  Newton's method gives
  x_n+1 = x_n - f(x_n)/f ^'(x_n)
         = x_n - (x_n^1/3)/((x_n^-2/3)/3) = -2x_n
  This means that no matter what the initial point, the next x value is twice the previous one (absolute value), and with each proceedings step, moves farther from the solution. This condition is called a diverging solution, and Newton's method obviously fails.