Apply function to rolling window

Question

Say I have a long list A of values (say of length 1000) for which I want to compute the std in pairs of 100, i.e. I want to compute std(A(1:100))

Answer 1

To minimize number of operations, you can exploit the fact that the standard deviation can be computed as a difference involving second and first moments,

and moments over a rolling window are obtained efficiently with a cumulative sum (using cumsum):

A = randn(1000,4); %// random data
N = 100; %// window size
c = size(A,2);
A1 = [zeros(1,c); cumsum(A)];
A2 = [zeros(1,c); cumsum(A.^2)];
S = sqrt( (A2(1+N:end,:)-A2(1:end-N,:) ...
    - (A1(1+N:end,:)-A1(1:end-N,:)).^2/N) / (N-1) ); %// result

Benchmarking

Here's a comparison against a loop based solution, using timeit. The loop approach is as in Dan's solution but adapted to the 2D case, exploting the fact that std works along each column in a vectorized manner.

%// File loop_approach.m
function S = loop_approach(A,N);
[n, p] = size(A);
S = zeros(n-N+1,p);
for k = 1:(n-N+1)
    S(k,:) = std(A(k:(k+N-1),:));
end

%// File bsxfun_approach.m
function S = bsxfun_approach(A,N);
[n, p] = size(A);
ind = bsxfun(@plus, permute(0:n:(p-1)*n, [3,1,2]), bsxfun(@plus, 0:n-N, (1:N).')); %'
S = squeeze(std(A(ind)));

%// File cumsum_approach.m
function S = cumsum_approach(A,N);
c = size(A,2);
A1 = [zeros(1,c); cumsum(A)];
A2 = [zeros(1,c); cumsum(A.^2)];
S = sqrt( (A2(1+N:end,:)-A2(1:end-N,:) ...
    - (A1(1+N:end,:)-A1(1:end-N,:)).^2/N) / (N-1) );

%// Benchmarking code
clear all
A = randn(1000,4); %// Or A = randn(1000,1);
N = 100;
t_loop   = timeit(@() loop_approach(A,N));
t_bsxfun = timeit(@() bsxfun_approach(A,N));
t_cumsum = timeit(@() cumsum_approach(A,N));
disp(' ')
disp(['loop approach:   ' num2str(t_loop)])
disp(['bsxfun approach: ' num2str(t_bsxfun)])
disp(['cumsum approach: ' num2str(t_cumsum)])
disp(' ')
disp(['bsxfun/loop gain factor: ' num2str(t_loop/t_bsxfun)])
disp(['cumsum/loop gain factor: ' num2str(t_loop/t_cumsum)])

Results

I'm using Matlab R2014b, Windows 7 64 bits, dual core processor, 4 GB RAM:

• 4-column case:

  loop approach:   0.092035
  bsxfun approach: 0.023535
  cumsum approach: 0.0002338
  
  bsxfun/loop gain factor: 3.9106
  cumsum/loop gain factor: 393.6526
  

• Single-column case:

  loop approach:   0.085618
  bsxfun approach: 0.0040495
  cumsum approach: 8.3642e-05
  
  bsxfun/loop gain factor: 21.1431
  cumsum/loop gain factor: 1023.6236
  

So the cumsum-based approach seems to be the fastest: about 400 times faster than the loop in the 4-column case, and 1000 times faster in the single-column case.