Creating folder pattern in Amazon s3 using informatica BDM

爷,独闯天下 提交于 2019-12-11 11:26:42

问题


We are implementing data lake. To load the landing layer s3 bucket, we use sqoop jobs which does the task of loading data into S3 landing layer bucket.From Amazon s3 landing layer bucket, we are using Informatica Developer tool to pull the files/objects and load it into sanitised layer s3 bucket path - s3://amazon.s3.bucket/YYYYMM/File1.txt( We are maintaining a path for each month load). So, for each month load we are going to create folder like structure in s3 bucket sanitised layer. Through Informatica developer tool, I would like to pass the s3 bucket path of the file and the file name dynamically using parameter file for both read & write into s3 and by doing this way the file path will be created automatically in Amazon s3 if the path doesn't exists in Amazon s3. I would like to know how do we could pass the details dynamically.

Any idea regarding this implementation in Informatica Developer tool would be really appreciated.

Thanks, Vijay

来源:https://stackoverflow.com/questions/53307892/creating-folder-pattern-in-amazon-s3-using-informatica-bdm

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!