How to build a Snowflake query to get these results

♀尐吖头ヾ 提交于 2020-07-22 05:40:19

问题


The below table (TMP_RN_TC) in query is a temp table which would be used to load the data into the final table. This table has to get the data from stage-table and the output of temp-table data needs to be stored in final table. Stage table will get data for 15-days in every run.

But the fact/final table should store all the data for the first run and then after only one day of data which would be changing (rest 14-days data would remain same). Since stage-table will hold even the duplicate data, temp-table should be able to remove those duplicates and load only data for the day from second run. Distinct didn't help. Below is the data and query:

For example, in the first run for 15-days, we get 30 records but in the second run, stage is getting 30-more records that is 60-records in stage now after 2nd run, but temp table should pick only 2-records since that would only have changed in the second run and rest 14-days(28-rows) data would be same.

This is the query I want to build:

1-row Data looks like:

{
  "location": "xyz",
  "metrics": [
    {
    "name": "traffic_in",
      "data": [
        {
          "group": {
            "start": "2020-07-05",
            "type": "date"
          },
          "index": 0,
          "next_level": [
            {"index": 0,
              "validity": "complete",
              "value": 1,
              "group": {
                "finish": "00:15",
                "start": "00:00",
                "type": "time"
              }
            }
          ]
        }
      ],
    }
  ],
}

Below is the snowflake-query:

create or replace TABLE TMP_RN_TC as
(select * from(
               select distinct
                   replace(D_NEXT : location , '"' , '')as  rn_loc_id,
                   mtr.value:name::VARCHAR as  metrics_name,
                   dta.value:group.start::DATE as metrics_event_date,
                   dta.value:index::numeric as metrics_date_index,
                   nxt.value:validity::VARCHAR as metrics_data_validity,
                   nxt.value:value::numeric as metrics_data_value,
                   nxt.value:group.start::time as metrics_data_start_tms,
                   nxt.value:index::numeric as metrics_time_index
           from STG_RN_TC stg,
           lateral flatten(input => stg.D_NEXT:metrics) mtr,
           lateral flatten(input => mtr.value:data) dta,
           lateral flatten(input => dta.value:next_level)nxt)
 ) ;

Note: from second run, I want only one day data to be coming in tmp-table which would be loaded in final table eventually.

来源:https://stackoverflow.com/questions/62952090/how-to-build-a-snowflake-query-to-get-these-results

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!