How can generate impulse as outliers on periodic or sequenced-based data for doing experiments via outliers detection methods?

Question

I want to carry out some experiments on some time-series data with the KM approach has been suggested in this paper. The problem is I don\'t access the data in the paper but I h

Answer 1

You can generate a signal with eg.: numpy Python module. And you can add your impulses to this signal (of course, if the dimensions are correct). I have written an example for your where I have generated a Sinus signal with numpy and I have added the impulses with the signal.unit_impulse() like in your question. I have added several comments to code for the better understanding.

Code:

import numpy as np
import matplotlib.pyplot as plt
import scipy.signal as signal

positive_impulses = signal.unit_impulse(200, [10, 50, 60])  # Generate positive impulses
negative_impulses = signal.unit_impulse(200, [80, 100, 150])*-1  # Generate negative impulses

# Generate the Sinus signal.
t = np.linspace(1, 200, 200)
x_sin_sig = np.sin(t / (2 * np.pi))
plt.subplot(4, 1, 1)
plt.plot(t, x_sin_sig)
plt.title("Signal")
plt.ylabel("Sin")


plt.subplot(4, 1, 2)
plt.plot(t, x_sin_sig + positive_impulses)  # Add the positive impulses to the original signal
plt.title("Signal with positive impulses")

plt.subplot(4, 1, 3)
plt.plot(t, x_sin_sig + negative_impulses)  # Add the negative impulses to the original signal
plt.title("Signal with negative impulses")

plt.subplot(4, 1, 4)
plt.plot(t, x_sin_sig + positive_impulses + negative_impulses)  # Add the both impulses to the original signal
plt.title("Signal with different impulses")

plt.tight_layout()
plt.show()

Output:

Note:

Probably more efficient to generate random impulses for your signal. You can do it with the random module.

import random

positive_impulses = signal.unit_impulse(200, random.sample(range(200), 5))  # Generate positive impulses
negative_impulses = signal.unit_impulse(200, random.sample(range(200), 5))*-1  # Generate negative impulses

The random.sample(range(200), 5) will return a list of 5 numbers selected from the range 0 to 200, without duplicates.

An example with random impulses:

Answer 2

Synthesizing anomalies in time-series can be done by considering the output as a combination of the normal signal and some noise. The most common noise model is additive noise, which means the the output is simply out = signal + noise. In your case you would scale the unit impulses to be of suitable amplitude before adding. In the example the amplitude of the impulsive noise seems around 15.

PS: Both amplitude, distance between impulses and width/shape of impulse should probably be random variables.

Answer 3

Thanks for the invitation. Regarding how to generate periodic outliers, I have no such experience till now. But as it is time series based, so I think we can start from Arima concept.

1. if you can find the p, d, q parameter, you already reach the data answer. though there is high computation cost. there is some reference in StackOverflow about how to find the p q d

2. to deal with the time series, the stationary is the most important thing in Arima mode. You can try to do 1st differential or 2nd differential. If the data set has just 1 feature, the outlier already showed out directly.

Good luck.

Hope it was helpful.

WY