Running shell script with Oozie

Question

I am trying to run a sh script through Oozie, but I am facing a problem:

Cannot run program \"script.sh\" (in directory \"/mapred/local/tas

Answer 1

Also make sure that the shell scripts are UNIX compliant. If these shell scripts were written in windows environment then it appends windows specific end of lines (EOL) and these scripts are not recognized by the oozie. So you will get "no such file or directory found" in oozie shell actions.

Answer 2

workflow.xml would look something like this

<workflow-app name="HiveQuery_execution" xmlns="uri:oozie:workflow:0.5">
<start to="shell-3c43"/>
<kill name="Kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<action name="shell-3c43">
    <shell xmlns="uri:oozie:shell-action:0.1">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <exec>/user/path/hivequery.sh</exec>
        <file>/user/path/hivequery.sh#hivequery.sh</file>
          <capture-output/>
    </shell>
    <ok to="End"/>
    <error to="Kill"/>
</action>
<end name="End"/>

Job.properties

jobTracker=xxxx.xxx.xxx.com:port
nameNode=hdfs://xxxx.xxx.xxx.com:port

better configure through UI, as suggested above

Answer 3

An Oozie shell action is executed on a random Hadoop node, i.e. not locally on the machine where the Oozie server is running. As Oleksii says, you have to make sure that your script is on the node that executes the job.

See the following complete examples of executing a shell action and an ssh action:

https://github.com/airawat/OozieSamples/tree/master/oozieProject/workflowShellAction https://github.com/airawat/OozieSamples/tree/master/oozieProject/workflowSshAction

Answer 4

Try to give full path for HDFS like

<exec>/user/nathalok/run.sh</exec>  
<file>/user/nathalok/run.sh#run.sh</file> 

and ensure that in job.properties the path is mentioned correctly for the library and workflow.xml

oozie.libpath=hdfs://server/user/oozie/share/lib/lib_20150312161328/oozie
oozie.wf.application.path=hdfs://bcarddev/user/budaledi/Teradata_Flow

Answer 5

This error is really ambiguous. Here are some issues that have helped me to solve this issue.

-If you are running oozie workflows on a kerberized cluster, make sure to authenticate by passing your Kerberos Keytab as a argument:

...
<shell>
  <exec>scriptPath.sh</exec>
  <file>scriptPath.sh</file>
  <file>yourKeytabFilePath</file>
</shell>
...

-In your shell File (scriptPath.sh), make sure ro remove first line shell reference.

#!usr/bin/bash

indeed, if this shell reference isn't deployed on all data nodes, this can lead to this error code.

Answer 6

if your shell file exist in your project correlated dir. then it's your shell file format cause this error. you need to convert the format from dos to linux using dos2linux:dos2linux xxxx.sh