Filling in NULLS with previous records - Netezza SQL

问题

I am using Netezza SQL on Aginity Workbench and have the following data:

id           DATE1              DATE2
1            2013-07-27         NULL
2            NULL               NULL
3            NULL               2013-08-02
4            2013-09-10         2013-09-23
5            2013-12-11         NULL
6            NULL               2013-12-19

I need to fill in all the NULL values in DATE1 with preceding values in the DATE1 field that are filled in. With DATE2, I need to do the same, but in reverse order. So my desired output would be the following:

id           DATE1              DATE2
1            2013-07-27         2013-08-02
2            2013-07-27         2013-08-02
3            2013-07-27         2013-08-02
4            2013-09-10         2013-09-23
5            2013-12-11         2013-12-19
6            2013-12-11         2013-12-19

I only have read access to the data. So creating Tables or views are out of the question

回答1:

How about this?

select
  id
  ,last_value(date1 ignore nulls) over (
    order by id
    rows between unbounded preceding and current row
  ) date1
  ,first_value(date2 ignore nulls) over (
    order by id
    rows between current row and unbounded following
  ) date2

You can manually calculate this as well, rather than relying on the windowing functions.

with chain as (
  select 
    this.*,
    prev.date1 prev_date1,
    case when prev.date1 is not null then abs(this.id - prev.id) else null end prev_distance,
    next.date2 next_date2,
    case when next.date2 is not null then abs(this.id - next.id) else null end next_distance
  from 
    Table1 this 
    left outer join Table1 prev on this.id >= prev.id
    left outer join Table1 next on this.id <= next.id
), min_distance as (
  select
    id,
    min(prev_distance) min_prev_distance,
    min(next_distance) min_next_distance
  from
    chain
  group by
    id
)
select
  chain.id,
  chain.prev_date1,
  chain.next_date2
from
  chain
  join min_distance on 
    min_distance.id = chain.id
    and chain.prev_distance = min_distance.min_prev_distance
    and chain.next_distance = min_distance.min_next_distance
order by chain.id

If you're unable to calculate the distance between IDs by subtraction, just replace the ordering scheme by a row_number() call.

回答2:

I think Netezza supports the order by clause for max() and min(). So, you can do:

select max(date1) over (order by date1) as date1,
       min(date2) over (order by date2 desc) as date2
 . . .

EDIT:

In Netezza, you may be able to do this with last_value() and first_value():

select last_value(date1 ignore nulls) over (order by id rows between unbounded preceding and 1 preceding) as date1,
       first_value(date1 ignore nulls) over (order by id rows between 1 following and unbounded following) as date2

Netezza doesn't seem to support IGNORE NULLs on LAG(), but it does on these functions.

回答3:

I've only tested this in Oracle so hopefully it works in Netezza:

Fiddle: http://www.sqlfiddle.com/#!4/7533f/1/0

select id,
       coalesce(date1, t1_date1, t2_date1) as date1,
       coalesce(date2, t3_date2, t4_date2) as date2
  from (select t.*,
               t1.date1 as t1_date1,
               t2.date1 as t2_date1,
               t3.date2 as t3_date2,
               t4.date2 as t4_date2,
               row_number() over(partition by t.id order by t.id) as rn
          from tbl t
          left join tbl t1
            on t1.id < t.id
           and t1.date1 is not null
          left join tbl t2
            on t2.id > t.id
           and t2.date1 is not null
          left join tbl t3
            on t3.id < t.id
           and t3.date2 is not null
          left join tbl t4
            on t4.id > t.id
           and t4.date2 is not null
         order by t.id) x
 where rn = 1

回答4:

Here's a way to fill in NULL dates with the most recent min/max non-null dates using self-joins. This query should work on most databases

select t1.id, max(t2.date1), min(t3.date2)
from tbl t1
join tbl t2 on t1.id >= t2.id
join tbl t3 on t1.id <= t3.id
group by t1.id

http://www.sqlfiddle.com/#!4/acc997/2

来源：https://stackoverflow.com/questions/32018805/filling-in-nulls-with-previous-records-netezza-sql

标签

apache-spark

Hive

hql