Compare two spectogram to find the offset where they match algorithm

Question

I record a daily 2 minutes radio broadcast from Internet. There\'s always the same starting and ending jingle. Since the radio broadcast exact time may vary from more or les

Answer 1

I wonder if you could use a Hough transform. You would start by cataloging each step of the opening sequence. Let's say you use 10 ms steps and the opening sequence is 50 ms long. You compute some metric on each step and get

1 10 1 17 5

Now go through your audio and analyze each 10 ms step for the same metric. Call this array have_audio

8 10 8 7 5 1 10 1 17 6 2 10...

Now create a new empty array that's the same length as have_audio. Call it start_votes. It will contain "votes" for the start of the opening sequence. If you see a 1, you may be in the 1st or 3rd step of the opening sequence, so you have 1 vote for the opening sequence starting 1 step ago and 1 vote for the opening sequence starting 3 steps ago. If you see a 10, you have 1 vote for the opening sequence starting 2 steps ago, a 17 votes for 4 step ago, and so on.

So for that example have_audio, your votes will look like

2 0 0 1 0 4 0 0 0 0 0 1 ...

You have a lot of votes at position 6, so there's a good chance the opening sequence starts there.

You could improve performance by not bothering to analyze the entire opening sequence. If the opening sequence is 10 seconds long, you could just search for the first 5 seconds.

Answer 2

There's a description of the algorithm used by the shazam service (which identifies a music given a short possibly noisy sample) here : http://www.ee.columbia.edu/~dpwe/papers/Wang03-shazam.pdf
From what I understood, the first thing done is to isolate peaks in the spectrogram (with some tweaks to assure an uniform coverage), which will give a "constellation" of pair of values (time;frequency) from the initial spectrogram. Once done, the sample constellation is compared to the constellation of the full track by translating a window of the sample length from the beginning to the end and counting the number of correlated points.
The paper then describes the technical solution they found to be able to do the comparison fast even with a huge collection of tracks.

Answer 3

Here is a good python package that does just this:

https://code.google.com/p/py-astm/

If you are looking for a specific algorithm, good search terms to use are "accoustic fingerprinting" or "perceptual hashing".

Here's another python package that could also be used:

http://rudd-o.com/new-projects/python-audioprocessing/documentation/manuals/algorithms/butterscotch-signatures

Answer 4

If you already know the jingle sequence, you could analyse the correlation with the sequence instead of the cross correlation between the full 15 minutes tracks.

To quickly calculate the correlation against the (short) sequence, I would suggest using a Wiener filter.

Edit: a Wiener filter is a way to locate a signal in a sequence with noise. In this application, we are considering anything that is "not jingle" as noise (question for the reader: can we still assume that the noise is white and not correlated?).

( I found the reference I was looking for! The formulas I remembered were a little off and I'll remove them now)

The relevant page is Wiener deconvolution. The idea is that we can define a system whose impulse response h(t) has the same waveform as the jingle, and we have to locate the point in a noisy sequence where the system has received an impulse (i.e.: emitted a jingje).

Since the jingle is known, we can calculate its power spectrum H(f), and since we can assume that a single jingle appears in a recorded sequence, we can say that the unknown input x(t) has the shape of a pulse, whose power density S(f) is constant at each frequency.

Given the knowledges above, you can use the formula to obtain a "jingle-pass" filter (as in, only signals shaped like the jingle can pass) whose output is highest when the jingle is played.