What is the fastest library/algorithm for calculating simple moving average? I wrote my own, but it takes too long on 330 000 items decimal dataset.
These days, the Math DotNet library has a class called RunningStatistics that will do this for you. If you want to do it over the last "X" items only, use MovingStatistics instead.
Both will calculate running averages, variance, and standard deviation, on the fly with one-pass only and without storing extra copies of the data.
I find the provide answers a bit to memory hungry, and slow, you asked for fast. Add 2 fields one to keep the running total and one for the times the value changed as average is the sum/count of a list of values. I added a Add method, however you can also just use variables in a method….
public class Sample
{
private decimal sum = 0;
private uint count = 0;
public void Add(decimal value)
{
sum += value;
count++;
}
public decimal AverageMove => count > 0 ? sum / count : 0;
}
to make it thread safe:
public class ThreadSafeSample
{
private decimal sum = 0;
private uint count = 0;
private static object locker = new object();
public void Add(decimal value)
{
lock (locker)
{
sum += value;
count++;
}
}
public decimal AverageMove => count > 0 ? sum / count : 0;
}
Your main problem is that you throw away too much information for each iteration. If you want to run this fast, you need to keep a buffer of the same size as the frame length.
This code will run moving averages for your whole dataset:
(Not real C# but you should get the idea)
decimal buffer[] = new decimal[period];
decimal output[] = new decimal[data.Length];
current_index = 0;
for (int i=0; i<data.Length; i++)
{
buffer[current_index] = data[i]/period;
decimal ma = 0.0;
for (int j=0;j<period;j++)
{
ma += buffer[j];
}
output[i] = ma;
current_index = (current_index + 1) % period;
}
return output;
Please note that it may be tempting to keep a running cumsum instead of keeping the whole buffer and calculating the value for each iteration, but this does not work for very long data lengths as your cumulative sum will grow so big that adding small additional values will result in rounding errors.
If the data is static, you can preprocess the array to make moving average queries very fast:
decimal[] GetCSum(decimal[] data) {
decimal csum[] = new decimal[data.Length];
decimal cursum = 0;
for(int i=0; i<data.Length; i++) {
cursum += data[i];
csum[i] = cursum;
}
return csum;
}
Now the moving average calculation is easy and fast:
decimal CSumMovingAverage(decimal[] csum, int period, int ii) {
if(period == 0 || ii <= period)
return -1;
return csum[ii] - csum[ii - period];
}
Here's how I tried it. But warning I'm a complete amateur so this may be completely wrong.
List<decimal> MovingAverage(int period, decimal[] Data)
{
decimal[] interval = new decimal[period];
List<decimal> MAs = new List<decimal>();
for (int i=0, i < Data.length, i++)
{
interval[i % period] = Data[i];
if (i > period - 1)
{
MAs.Add(interval.Average());
}
}
return MAs;
}
Should return a list of decimals containing the moving averages for your data.
This is MA I'm using in my app.
double[] MovingAverage(int period, double[] source)
{
var ma = new double[source.Length];
double sum = 0;
for (int bar = 0; bar < period; bar++)
sum += source[bar];
ma[period - 1] = sum/period;
for (int bar = period; bar < source.Length; bar++)
ma[bar] = ma[bar - 1] + source[bar]/period
- source[bar - period]/period;
return ma;
}
Once you have it calculated for the whole data series, you can grab a particular value instantly.