How to change value in an oozie job coordinator?

一曲冷凌霜 提交于 2019-12-11 10:53:58

问题


I have a mapreduce job which is scheduled by an oozie coordinator and runs every 4 hours. This mapreduce job takes a parameter, let's say k, whose value is set in the job.config file. I'd like to know if I change the value of this parameter between two runs, does it pick the updated (new) value or it sticks to the original (old) value?


回答1:


if the job is in runing mode, it will stick to Old parameter it self, and if the job is in waiting to schedule run, then it will take the latest value :).




回答2:


Actually, there is a devious way to "dynamically" fetch a parameter value at run time:

  1. insert a dummy Shell Action at the beginning of the Workflow, with <capture-output/> option set
  2. in the shell script, just download a properties file from HDFS and dump it to STDOUT
  3. the "capture-output" option tells Oozie to parse STDOUT into a Map (i.e. a key/value list)
  4. then use an E.L. function to retrieve the appropriate value(s) in the next Actions

${wf:actionData("DummyShellAction")["some.key"]}

http://oozie.apache.org/docs/4.0.0/WorkflowFunctionalSpec.html#a4.2.6_Hadoop_Jobs_EL_Function



来源:https://stackoverflow.com/questions/31521957/how-to-change-value-in-an-oozie-job-coordinator

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!