queue with time stamped elements within a time period

Question

I want to store in a queue, datastructure does not matter, only the elements that I have inserted within say last 5 minutes from current time. Anything older should get removed

Answer 1

You can use a Priority Queue with timestamps as your keys. So that when you call Peek() you always get the oldest timestamp still in the queue. Then each time you go to query for the number of items inside your window size: you cleanup the items outside your window and return the number of items still in the Priority queue.

For example:

public class CountInWindow {

    /**
     * Adding a main just for testing 
     * @param args
     * @throws InterruptedException 
     */
    public static void main(String[] args) throws InterruptedException {
        System.out.println("test started");
        CountInWindow test = new CountInWindow(5000); //5 seconds for testing
        test.debug = true;
        test.insertTimeStamp(System.currentTimeMillis());
        Thread.sleep(100);//sleep 
        test.insertTimeStamp(System.currentTimeMillis());
        Thread.sleep(100);//sleep 
        test.insertTimeStamp(System.currentTimeMillis());
        Thread.sleep(100);//sleep 
        test.insertTimeStamp(System.currentTimeMillis());
        Thread.sleep(5040);//sleep 5 secs
        test.insertTimeStamp(System.currentTimeMillis());
        Thread.sleep(100);//sleep 
        test.insertTimeStamp(System.currentTimeMillis());
        System.out.println(test.getWindowCount()); //Should be 2 not 6.
        System.out.println("test done");
    }

    java.util.PriorityQueue<Long> window;
    public static final long FIVE_MINS_IN_MS = 300000l;
    public final long WINDOW_SIZE;
    public boolean debug = false;

    //Constructor which defaults to 5mins
    public CountInWindow(){
        WINDOW_SIZE = FIVE_MINS_IN_MS;
        window = new java.util.PriorityQueue<Long>();
    }
    //Constructor for any size window
    public CountInWindow(long windowSize){
        WINDOW_SIZE = windowSize;
        window = new java.util.PriorityQueue<Long>();
    }
    /**
     * Add a new timestamp to the window's queue
     * @param ts
     */
    public void insertTimeStamp(long ts){
        window.add(ts);
    }
    /**
     * Clean up items outside the window size and then return the count of times still in the window.
     * @return A count of timestamps still inside the 5 mins window.
     */
    public int getWindowCount(){
        long currTime = System.currentTimeMillis();
        //Clean out old Timestamps
        while((currTime - window.peek().longValue()) > WINDOW_SIZE){
            long drop = window.remove().longValue();
            if(debug)System.out.println("dropping item:" + drop);
        }
        return window.size();
    }
}

Answer 2

I have implement a FadingLinkedList like

public class FadingLinkedList<E> {

private transient Entry<E> header = new Entry<E>(null, null);

/**
 * ms
 */
private long livingTime;

/**
 * Constructs FadingLinkedList with elements of living time livingTime in
 * milliseconds
 */
public FadingLinkedList(long livingTime) {
    this.livingTime = livingTime;
}

/**
 * remove all faded elements,
 *
 * @return the count of not faded
 */
public synchronized int removeFaded() {
    long now = System.nanoTime();
    int count = 0;
    Entry<E> prev = header;// the last living Entry in the loop
    for (Entry<E> e = header.next; e != null; e = e.next) {
        if (TimeUnit.NANOSECONDS.toMillis(now - e.birthTime) >= livingTime) {
            // cut off this list here.
            prev.next = null;
            break;
        }
        count++;
        prev = e;
    }
    return count;
}

/**
 * Returns the number of elements that not faded.
 */
public int size() {
    return removeFaded();
}

public synchronized void push(E e) {
    Entry<E> newEntry = new Entry<E>(e, header.next);
    header.next = newEntry;
}

private static class Entry<E> {
    E element;
    Entry<E> next;
    long birthTime;

    Entry(E element, Entry<E> next) {
        this.element = element;
        this.next = next;
        this.birthTime = System.nanoTime();
    }
}

public synchronized void clear() {
    header.next = null;
}

public synchronized int getAndClear() {
    int size = size();
    clear();
    return size;
}

}

Answer 3

In what language? Is the queue persistent or in-memory?

If you need this behavior in Java, you can use a DelayedQueue, and have a separate thread calling queue.take() continuously in a tight loop to drain out expired items. queue.size() will then give you the size of remaining unexpired items in the queue. This requires that the items you put in the DelayedQueue implement the Delayed interface and return the value 5 minutes to the .getDelay() method.