Understanding scipy integrate's internal behavior

Question

I am trying to understand what scipy.integrate is doing internally. Namely, it seems that something weird and inconsistent is happening.

How get it

Answer 1

All solvers that are better than the standard fixed-step RK4 method use a variable step size. Treating the solver as a black-box, one can not know what internal step sizes are used.

What is known, however, is that the explicit one-step methods have multiple stages, at least equal to their order, that each comprise a call to the ODE function at a point close to, but not necessarily on the solution trajectory. Implicit methods may have less stages than the order, but require an iterative approach to the solution of the implicit step equations.

The Dormand-Prince 45 method has 7 stages, where the last stage is also the first of the next step, so in the long-time average 6 evaluations per step. This is what you see in the ode(dopri) method.

INTERNAL t =   0.00000000
INTERNAL t =   0.01000000
INTERNAL t =   0.00408467
INTERNAL t =   0.00612700
INTERNAL t =   0.01633866
INTERNAL t =   0.01815407
INTERNAL t =   0.02042333
INTERNAL t =   0.02042333
INTERNAL t =   0.03516563
INTERNAL t =   0.04253677
INTERNAL t =   0.07939252
INTERNAL t =   0.08594465
INTERNAL t =   0.09413482
INTERNAL t =   0.09413482
Outside integrate t =   0.09413482
...

There one can see that the minimal step of the scipy method consists of 2 DoPri steps. In the sequence of the first step, the first evaluation is just probing if the initial step size is appropriate, this is only done once. All the other step points are at the prescribed times t_n+c_i*dt where c=[0,1/5,3/10,4/5,8/9,1,1].

You can get proper single steps with the new classes that are the steppers for the new interface solve_ivp. Take care that the default tolerances are here much looser than in the ode(dopri) case, probably following the Matlab philosophy of generating "good enough" plots with minimal effort. For RK45 this can look like

simulator = RK45(myODE, t0, [1,1], t1, atol=6.8e-7, rtol=2.5e-8)

t = simulator.t

while t < t1:
    simulator.step() 
    t = simulator.t
    x = simulator.y
    print(f'Outside integrate t = {t:12.8f}')
    print(f'x1 = {x[0]:12.10f}, err = {x[0]-1/(1+t):8.6g}')

This uses slightly different internal steps, but, as said, has a "true" single-step output.

INTERNAL t =   0.00000000
INTERNAL t =   0.01000000
INTERNAL t =   0.00408223
INTERNAL t =   0.00612334
INTERNAL t =   0.01632891
INTERNAL t =   0.01814323
INTERNAL t =   0.02041114
INTERNAL t =   0.02041114
Outside integrate t =   0.02041114
x1 = 0.9799971436, err = 5.2347e-13
INTERNAL t =   0.04750541
INTERNAL t =   0.06105254
INTERNAL t =   0.12878821
INTERNAL t =   0.14083011
INTERNAL t =   0.15588248
INTERNAL t =   0.15588248
Outside integrate t =   0.15588248
x1 = 0.8651399668, err = 1.13971e-07
...

If you have an input that is a step function, or a zero-order hold, the most expedient solution would be to loop over the steps and initialize one RK45 object per step with the step segment as integration boundaries. Save the last value as initial value for the next step. Perhaps also the last step size as initial step size in the next step.

Directly using a step function inside the ODE function is inefficient, as the step size controller expects a very smooth ODE function for an optimal step size sequence. At jumps that is grossly violated and can lead to stark local reductions in the step size, and accordingly an increased number of function evaluations.