How do I process billions of records in ADLS data using spark less time

后端 未结 0 1048
面向向阳花
面向向阳花 2021-01-26 06:26

I have data that has avg of 1-3billion rows per month. I am doing simple sql query to filter out data basis date(ie part of partition) and few more conditions. But this takes 6-

相关标签:
回答
  • 消灭零回复
提交回复
热议问题