Which numerical method to use for ODE?

Question

In practice what is the most common way to numerically estimate $y(t)$ (possibly using a series expansion) in the ODE with initial conditions, $$ y'(t) = f(t,y(t)), \qquad y(t_0)=y_0 $$ Wikipedia has a page on this but it is not clear which of the methods is most used in practice http://en.wikipedia.org/wiki/Numerical_methods_for_ordinary_differential_equations

As additional information, in the problem that I am dealing with $f(t,y(t))$ is also estimated so I prefer a method that is robust but am interested to hear pros and cons of other methods.

Answer 1

There are various ways of doing so. For sure the simplest is Euler's method. So simple that the global error is very big and of not much use. However it is important to use it as it explains in a simple way how numerical solutions to ODE work and is a good way to check that more advanced methods work.

The method used the more often by the scientific community is the Runge-Kutta of 4th order, also called the "classic RK". It has small global error and is still easy to implement. This Wikipedia page explains well the equations involved and as you can see, it remains simple.

There are other RK methods of higher order but I can't find the equations. I saw the equations for 8th order and they were horrid to implement. Matlab claims is has a 12th order available.

Other techniques you can use are the PECE methods (Predict Estimate Correct Estimate). You can have a look here to see what it looks like but this is more advanced. On the up side, you can obtain an incredibly small global error.

Overall, $RK4$ is the best compromise to solve an ODE of your form as it's quick and robust.

Answer 2

This field is huge and I suggest picking up a good book to learn it thoroughly. But I can give you an outline of the types of methods with when/why they are used so you can pick the right method for your problem.

The best method is very dependent on the problem. For "easy problems" (to define later) the best methods are Runge-Kutta problems. These take just a simple loop to implement where you add things together with coefficients determined by theory. RK4 is the fourth order Runge-Kutta method and is order 4, which means if you halve the timestep you get multiply the error by $2^{-4}$. The reason why it is the most popular is because after 4th order you begin to have diminishing returns: the number of calculations to increase the order by 1 begins to grow exponentially. So most methods use something around order 4. The most popular method adjusts its timestep to reduce error and is called the Dommand-Prince Method, which you can think of as just a really smart RK5. In MATLAB this is ode45.

However, error is not the only problem you can encounter. Numerical methods can be what's known as unstable. The idea is that, even if the ODE is stable (goes to a steady state), the numerical method can blow up! The stability of the ODE is directly related to the eigenvalues of the Jacobian (in the 1D case, this is the size of the derivative of $f$). If these values are large, then the equation is "stiff" and so Runge-Kutta solvers would need a really small timestep to not blow up. For these types of equations people normally use what's known as implicit solvers. These tend to have very very good stability. They step forward using implicit equations which have to be solved via Newton's method at every time step. Thus it's clear that is a lot less efficient than just looping through and plugging in numbers, but for stiff equations this is required. In MATLAB routines similar to this are the stiff solvers like ode15s.

Then there's a lot more you can do. In the implicit equation, instead of solving it directly via Newton's method, you can "guess" values with an explicit method (like Runge-Kutta) and then use those guesses in the implicit equation. These are predictor-corrector methods which don't have as much stability as the implicit methods, but have more than the Runge-Kutta methods and are easier to solve than the implicit methods. So they work best for some kind of "medium-high stiffness".

Of course, there's a lot more you can do, but in general these tend to be the main methods.