Running parallel jobs in talend

余生长醉 提交于 2019-12-03 21:11:29

问题


I have a situation where i need to run five different child jobs in talend in parallel. Problem is that, in my select query i would be getting five different ID's and then for each particular id , i need to run five different jobs. Problem with tparrallelize component is that , it does not allow me to pass context variables to each sub job, i.e id in this particular case.

select id from table limit 5; ----> five different instance of same job with different id as parameter

Any help would be highly appreciated

thanks


回答1:


I'm not sure if I properly understand what you're doing here but if you were to break out each of those IDs and store them as 5 separate context variables then each job could access their own context variable with the right ID stored for each of them and use that.

So I would start with your database input component (just select the IDs you want) and feed that into a tFlowToIterate. Connect this via an iterate flow into a tFixedFlowInput component and create 2 fields in your schema, "key" and "value". Use the inline table to specify that "key" should be ((Integer)globalMap.get("tFlowToIterate_1_CURRENT_ITERATION")) and "value" should be ((String)globalMap.get("row1.SupplierPartNumber")).

I'd then throw this into a tMap component where I'd put "ContextNumber" + row2.key into the mapped key column just to make it a bit more obvious than the iteration number as your context and then feed that directly into a tContextLoad.

From there you can OnSubjobOK to your tParallelize component and link all your jobs together. In each job configure the jobs to use the appropriate context variable.



来源:https://stackoverflow.com/questions/21309946/running-parallel-jobs-in-talend

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!