问题
We are implementing data lake. To load the landing layer s3 bucket, we use sqoop jobs which does the task of loading data into S3 landing layer bucket.From Amazon s3 landing layer bucket, we are using Informatica Developer tool to pull the files/objects and load it into sanitised layer s3 bucket path - s3://amazon.s3.bucket/YYYYMM/File1.txt( We are maintaining a path for each month load). So, for each month load we are going to create folder like structure in s3 bucket sanitised layer. Through Informatica developer tool, I would like to pass the s3 bucket path of the file and the file name dynamically using parameter file for both read & write into s3 and by doing this way the file path will be created automatically in Amazon s3 if the path doesn't exists in Amazon s3. I would like to know how do we could pass the details dynamically.
Any idea regarding this implementation in Informatica Developer tool would be really appreciated.
Thanks, Vijay
来源:https://stackoverflow.com/questions/53307892/creating-folder-pattern-in-amazon-s3-using-informatica-bdm