oozie-coordinator

How to auto rerun of failed action in oozie?

北战南征 提交于 2019-12-06 08:58:41
问题 How can I re-run any action which was failed in the workflow automatically? I know the way to rerun manually from command line or thorough hue. $oozie job -rerun ... Is there any parameter we can set or provide in workflow to retry automatically when action fails? 回答1: Most of the time, when an action fails in the Oozie workflow, you need to debug and fix the error and rerun the workflow. But there are times, when you want Oozie to retry the action after an interval, for fixed number of times

How to force coordinator action materialization at specific frequency?

百般思念 提交于 2019-12-06 03:52:32
I would like to know if it is possible/how to force a coordinator to materialize or instantiate workflow at regular intervals even if previous instantiated workflow are not done yet. Let me explain: I have a simple coordinator looking like this: <coordinator-app name="myApp" frequency="${coord:hours(3)}" start="2015-01-01T0:00Z" end="2016-01-01T00:00Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.4"> <action> <workflow> <app-path>${myPath}/workflow.xml</app-path> </workflow> </action> </coordinator-app> The frequency is set to 3 hours. Every 3 hours, I expect the coordinator to "materialize"

How oozie handle dependencies?

佐手、 提交于 2019-12-05 18:35:10
问题 I have several questions about oozie 2.3 share libraries: Currently, I defined the share libraries in our coordinator.properties: oozie.use.system.libpath=true oozie.libpath=<hdfs_path> Here are my questions: When share libraries are copied to other data node and how many data node will get share libraries? Are the share libraries copied to other data node based on number of wf in a coordinator job or they are only copied once per coordinator job? 回答1: Adding entries to the oozie.libpath

How to execute parallel jobs in oozie

别说谁变了你拦得住时间么 提交于 2019-12-05 12:49:08
I have a shell script in HDFS. I have scheduled this script in oozie with the following workflow. Workflow: <workflow-app name="Shell_test" xmlns="uri:oozie:workflow:0.5"> <start to="shell-8f63"/> <kill name="Kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <action name="shell-8f63"> <shell xmlns="uri:oozie:shell-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <exec>shell.sh</exec> <argument>${input_file}</argument> <env-var>HADOOP_USER_NAME=${wf:user()}</env-var> <file>/user/xxxx/shell_script/lib

How to auto rerun of failed action in oozie?

假如想象 提交于 2019-12-04 14:57:36
How can I re-run any action which was failed in the workflow automatically? I know the way to rerun manually from command line or thorough hue. $oozie job -rerun ... Is there any parameter we can set or provide in workflow to retry automatically when action fails? Most of the time, when an action fails in the Oozie workflow, you need to debug and fix the error and rerun the workflow. But there are times, when you want Oozie to retry the action after an interval, for fixed number of times before failing the workflow. You can specify the retry-max and retry-interval in the action definition.

How to check whether the file exist in HDFS location, using oozie?

醉酒当歌 提交于 2019-12-01 22:24:36
问题 How to check whether a file in HDFS location is exist or not, using Oozie? In my HDFS location I will get a file like this test_08_01_2016.csv at 11PM , on a daily basis. I want check whether this file exist after 11.15 PM. I can schedule the batch using a Oozie coordinator job. But how can I validate if the file exists in HDFS? 回答1: you can use EL expression in oozie like: <decision name="CheckFile"> <switch> <case to="nextOozieTask"> ${fs:exists('/path/test_08_01_2016.csv')} <!--do note the

sqoop job shell script execute parallel in oozie

烈酒焚心 提交于 2019-12-01 13:56:01
I have a shell script which executes sqoop job . The script is below. !#/bin/bash table=$1 sqoop job --exec ${table} Now when I pass the table name in the workflow I get the sqoop job to be executed successfully. The workflow is below. <workflow-app name="Shell_script" xmlns="uri:oozie:workflow:0.5"> <start to="shell"/> <kill name="Kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <action name="shell_script"> <shell xmlns="uri:oozie:shell-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <exec>sqoopjob

Error on running multiple Workflow in OOZIE-4.1.0

谁说我不能喝 提交于 2019-11-28 00:06:48
I installed oozie 4.1.0 on a Linux machine by following the steps at http://gauravkohli.com/2014/08/26/apache-oozie-installation-on-hadoop-2-4-1/ hadoop version - 2.6.0 maven - 3.0.4 pig - 0.12.0 Cluster Setup - MASTER NODE runnig - Namenode, Resourcemanager ,proxyserver. SLAVE NODE running -Datanode,Nodemanager. When I run single workflow job means it succeeds. But when I try to run more than one Workflow job i.e. both the jobs are in accepted state Inspecting the error log, I drill down the problem as, 014-12-24 21:00:36,758 [JobControl] INFO org.apache.hadoop.ipc.Client - Retrying connect

Oozie not sending SLA email alerts

♀尐吖头ヾ 提交于 2019-11-27 08:47:51
问题 I used this link from oozie documentation to setup SLAs for my oozie workflow. I then scheduled a job which ran longer than the defined SLAs. However, I am not getting any email alerts for SLA miss from oozie. Any idea on how I should debug it? Thank you! 来源: https://stackoverflow.com/questions/57281650/oozie-not-sending-sla-email-alerts

Error on running multiple Workflow in OOZIE-4.1.0

試著忘記壹切 提交于 2019-11-26 23:23:24
问题 I installed oozie 4.1.0 on a Linux machine by following the steps at http://gauravkohli.com/2014/08/26/apache-oozie-installation-on-hadoop-2-4-1/ hadoop version - 2.6.0 maven - 3.0.4 pig - 0.12.0 Cluster Setup - MASTER NODE runnig - Namenode, Resourcemanager ,proxyserver. SLAVE NODE running -Datanode,Nodemanager. When I run single workflow job means it succeeds. But when I try to run more than one Workflow job i.e. both the jobs are in accepted state Inspecting the error log, I drill down the