Interpolate missing values in a MySQL table

Question

I have some intraday stock data saved into a MySQL table which looks like this:

+----------+-------+
| tick     | quote |
+----------+-------+
| 08:00:10 |           


        

Answer 1

The main problem here is reference to main query in subquery t2.tick < t1.tick. Because of this you can't simply wrap the subquery in a another subquery.

If this is one time query and there is not so many data, you can do something like that:

UPDATE `table` AS t1
SET quote = (SELECT quote FROM (SELECT quote, tick FROM `table` AS t2 WHERE t2.quote IS NOT NULL) as t3 WHERE t3.tick < t1.tick ORDER BY t3.tick DESC LIMIT 1)
WHERE quote IS NULL

But really, really don't use that as it will be probably to slow. On each null quote, this query selects all data from table table and then from results it gets desired row.

Answer 2

I would create a (temporary) table with the same layout as your table and run the following two queries:

Insert all interpolations into the temp_stock table

INSERT INTO temp_stock (tick, quote) 
  SELECT s2.tick
         , (s1.quote + s3.quote) /2 as quote
  FROM stock 
  INNER JOIN stock s1 ON (s1.tick < s2.tick) 
  INNER JOIN stock s3 ON (s3.tick > s2.tick) 
  WHERE s2.quote IS NULL
  GROUP BY s2.tick
  HAVING s1.tick = MAX(s1.tick), s3.tick = MIN(s3.tick)  

Update the stock table with the temp values

  UPDATE stock s 
  INNER JOIN temp_stock ts ON (ts.tick = s.tick) SET s.quote = ts.quote

It does use a temp table (make sure it's a memory table for speed), but it doesn't need a cursor.

Answer 3

This should work:

SET @prev = NULL;

UPDATE   ticks
SET      quote= @prev := coalesce(quote, @prev)
ORDER BY tick;

BTW the same trick works for reading:

SELECT   t.tick, @prev := coalesce(t.quote, @prev)
FROM     ticks t
JOIN     (SELECT @prev:=NULL) as x -- initializes @prev
ORDER BY tick