The Newton method for systems of equations

Combining equations \(\eqref{eq:system_of_equations}\) and \(\eqref{eq:system_of_equations_jacobian}\), we get \[\begin{align} \require{physics} \vb{f}(\vb{x}_0) + \vb{J}(\vb{x}_0) (\vb{x} - \vb{x}_0) &\approx \vb{0} \\ \vb{J}(\vb{x}_0) (\vb{x} - \vb{x}_0) & \approx - \vb{f}(\vb{x}_0) \label{eq:system_of_equations_approx} \thinspace , \end{align}\] which means that if we have an initial \(\vb{x}_0\) (a guess for the solution), we can in principle calculate an improved guess \(\vb{x}\). Newton’s method thus goes as follows (Burden and Faires 2011):

1. Choose an initial guess \(\vb{x}_0\)
2. In the \((i+1)\)-th iteration, solve \(\eqref{eq:system_of_equations_approx}\) for \(\Delta \vb{x}_{i+1} = \vb{x}_{i+1} - \vb{x}_i\).
3. Update \[\begin{equation} \vb{x}_{i+1} = \vb{x}_i + \Delta \vb{x}_{i+1} \end{equation}\]
4. Recalculate the Jacobian and the vector field at \(\vb{x}_{i+1}\).
5. Repeat steps \(\ref{en:newton:first}\) to \(\ref{en:newton:last}\) until convergence is achieved: \[\begin{equation} || \Delta \vb{x}_{i+1} || < \epsilon \thinspace . \end{equation}\]

References

Burden, Richard L., and J. Douglas Faires. 2011. Numerical Analysis. Ninth. Brooks/Cole.