问题
The below table (TMP_RN_TC) in query is a temp table which would be used to load the data into the final table. This table has to get the data from stage-table and the output of temp-table data needs to be stored in final table. Stage table will get data for 15-days in every run.
But the fact/final table should store all the data for the first run and then after only one day of data which would be changing (rest 14-days data would remain same). Since stage-table will hold even the duplicate data, temp-table should be able to remove those duplicates and load only data for the day from second run. Distinct didn't help. Below is the data and query:
For example, in the first run for 15-days, we get 30 records but in the second run, stage is getting 30-more records that is 60-records in stage now after 2nd run, but temp table should pick only 2-records since that would only have changed in the second run and rest 14-days(28-rows) data would be same.
This is the query I want to build:
1-row Data looks like:
{
"location": "xyz",
"metrics": [
{
"name": "traffic_in",
"data": [
{
"group": {
"start": "2020-07-05",
"type": "date"
},
"index": 0,
"next_level": [
{"index": 0,
"validity": "complete",
"value": 1,
"group": {
"finish": "00:15",
"start": "00:00",
"type": "time"
}
}
]
}
],
}
],
}
Below is the snowflake-query:
create or replace TABLE TMP_RN_TC as
(select * from(
select distinct
replace(D_NEXT : location , '"' , '')as rn_loc_id,
mtr.value:name::VARCHAR as metrics_name,
dta.value:group.start::DATE as metrics_event_date,
dta.value:index::numeric as metrics_date_index,
nxt.value:validity::VARCHAR as metrics_data_validity,
nxt.value:value::numeric as metrics_data_value,
nxt.value:group.start::time as metrics_data_start_tms,
nxt.value:index::numeric as metrics_time_index
from STG_RN_TC stg,
lateral flatten(input => stg.D_NEXT:metrics) mtr,
lateral flatten(input => mtr.value:data) dta,
lateral flatten(input => dta.value:next_level)nxt)
) ;
Note: from second run, I want only one day data to be coming in tmp-table which would be loaded in final table eventually.
来源:https://stackoverflow.com/questions/62952090/how-to-build-a-snowflake-query-to-get-these-results