Azure Data Factory : querying _ts (epoch) field in CosmosDB source

孤街浪徒 提交于 2019-12-24 01:07:29

问题


I am trying to copy data from Azure CosmosDB into Azure SQL Database.

In the Azure CosmosDB source, the query is as follows :

select * from c where c.DefinitionTime >= '@{formatDateTime(pipeline().parameters.windowStart, 'yyyy-MM-ddTHH:mm:ssZ' )}' AND c.DefinitionTime < '@{formatDateTime(pipeline().parameters.windowEnd, 'yyyy-MM-ddTHH:mm:ssZ' )}'

In CosmosDB, "DefinitionTime" is stored as a string. To the run the above query, I have to update "DefinitionTime" with range-indexing which is one possible solution.

Another possible solution to is to use the _ts field which is available on all CosmosDB documents. _ts represents when the document was created or last updated in epoch time.

So basically the question is how to convert pipeline().parameters.windowStart to epoch time. The "formatDateTime" function's second parameter that takes in format doesn't seem to have a way of asking for epoch format. Additionally, there is no way of doing datetime arithmetic to convert to epoch manually.

来源:https://stackoverflow.com/questions/49164744/azure-data-factory-querying-ts-epoch-field-in-cosmosdb-source

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!