oozie-coordinator

问题 I want to run workflow based on availability of Control files for previous date. Date format in my directory is ${basePath}/YYYYMMdd/00/_Complete.I want to check the _Complete file inside my 00. My Job will run daily on the previous day data. I tried the options provided in similar questions But still not working. When I am testing it for same day data with below value for instance , it is working But not with (-1) option. Is there any restriction on URI-TEMPLATE formats, meaning do we need

问题 I want to run workflow based on availability of Control files for previous date. Date format in my directory is ${basePath}/YYYYMMdd/00/_Complete.I want to check the _Complete file inside my 00. My Job will run daily on the previous day data. I tried the options provided in similar questions But still not working. When I am testing it for same day data with below value for instance , it is working But not with (-1) option. Is there any restriction on URI-TEMPLATE formats, meaning do we need

问题 I want to run workflow based on availability of Control files for previous date. Date format in my directory is ${basePath}/YYYYMMdd/00/_Complete.I want to check the _Complete file inside my 00. My Job will run daily on the previous day data. I tried the options provided in similar questions But still not working. When I am testing it for same day data with below value for instance , it is working But not with (-1) option. Is there any restriction on URI-TEMPLATE formats, meaning do we need

问题 I'm not sure to use the good tool for what I want. I have a lot of workflows which can be dependent or not. Exemple : /workflow1 /workflow.xml /job.properties /workflow2 /workflow.xml /job.properties .... I thought that we can have a corrdinator which can launch (with some data conditions) all the workflow. But I begin to think that is not the good practice. Should we have one coordinator per workflow with all the conditions of executions + one bundle who launch all the coodinator ? like that

问题 I'm not sure to use the good tool for what I want. I have a lot of workflows which can be dependent or not. Exemple : /workflow1 /workflow.xml /job.properties /workflow2 /workflow.xml /job.properties .... I thought that we can have a corrdinator which can launch (with some data conditions) all the workflow. But I begin to think that is not the good practice. Should we have one coordinator per workflow with all the conditions of executions + one bundle who launch all the coodinator ? like that

问题 I am working on automating Pig jobs using oozie in hadoop cluster. I was able to run a sample pig script from oozie but my next requirement is to run a pig job where the pig script recieves it's input parameters from a shell script. Please share your thoughts 回答1: UPDATE: OK make the original question clear, how can you pass a parameter form a shell script output. Here's the working example: WORKFLOW.XML <workflow-app xmlns='uri:oozie:workflow:0.3' name='shell-wf'> <start to='shell1' />

问题 How to solution when error file doesnt exist in setting coordinator oozie: I have error in log coodinator: Pig logfile dump: Backend error message Error: java.io.FileNotFoundException: File does not exist: /user/hdfs/jay/part-0.tmp settingan coordinator: <coordinator-app name="tes-ng" frequency="${coord:minutes(15)}" start="2015-12-07T10:30+0700" end="2017-02-28T23:00+0700" timezone="Asia/Jakarta" xmlns="uri:oozie:coordinator:0.1" xmlns:sla="uri:oozie:sla:0.1"> <controls> <execution>LAST_ONLY

问题 I have a shell script which executes sqoop job . The script is below. !#/bin/bash table=$1 sqoop job --exec ${table} Now when I pass the table name in the workflow I get the sqoop job to be executed successfully. The workflow is below. <workflow-app name="Shell_script" xmlns="uri:oozie:workflow:0.5"> <start to="shell"/> <kill name="Kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <action name="shell_script"> <shell xmlns="uri:oozie:shell

问题 I want to move files in S3 using AWS oozie. I want to run aws s3 mv s3://temp/*.zip s3://temp/processed_files/. --recursive How I can do this in oozie? EDIT 1 2015-11-12 10:18:55,758 WARN ShellActionExecutor:542 - USER[hadoop] GROUP[-] TOKEN[] APP[rad_workflow] JOB[0000118-151029144311676-oozie-oozi-W] ACTION[0000118-151029144311676-oozie-oozi-W@sh] Launcher exception: Cannot run program "move.sh" (in directory "/mnt1/yarn/usercache/hadoop/appcache/application_1446129655727_0421/container

问题 I'm using HDP2.6. where is installed oozie 4.2. and Spark2. After I tracked Hortonworks guide on this site: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_spark-component-guide/content/ch_oozie-spark-action.html for adding libs for Spark2 in 4.2. version of Oozie. After I submit the job with this add-on: oozie.action.sharelib.for.spark=spark2 The error I'm getting is this: 2017-07-19 12:36:53,271 WARN SparkActionExecutor:523 - SERVER[] USER[admin] GROUP[-] TOKEN[] APP[Workflow2]