Oozie SSH Action

Question

Oozie SSH Action Issue:

Issue: We are trying to run few commands on a particular host machine of our cluster. We chose SSH Action for the same. We have been fa

Answer 1

I just hit a similar problem. I had a case where I could run as USER:

ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 USER@1.2.3.4 mkdir -p oozie-oozi/0000000-131008185935754-oozie-oozi-W/action1--ssh/

by hand on the command line and it worked, but when launched via Oozie as USER it failed.

The reason, in my case, it failed is that I set up passwordless ssh between USER on the oozie server and USER on the remote machine. What one needs to do is set up passwordless ssh between oozie on the oozie server and USER on the remote machine. In other words, su to oozie on the oozie server and run the above command by hand. If it fails, it will fail in Oozie. If it works, then it should work in Oozie (assuming all else is correct, like dir permissions, etc.)

Take a look at what user your oozie server is running as:

ps -ef | grep oozie

Whatever user that is needs passwordless ssh to USER on the remote machine.

Answer 2

After following all the above suggestion

oozie:x:488:487:Oozie User:/var/lib/oozie:/bin/false 

and change it to

oozie:x:488:487:Oozie User:/var/lib/oozie:/bin/bash

Just try these steps:

1. Create a password-less communication use below process:

   sudo su - oozie
   oozie@localhost: ssh-keygen -t dsa
   

   copy the public key generated to your local remote server like apps@XXXXXXX

2. try ssh apps@XXXXXXX, you will login to remote without error

3. go to HUE and select SSH action and give your BASH command like bash -x yourscript parameter
4. save
5. submit

Answer 3

Whatever quux00 has answered is right. I am just adding few points to that. As the command ssh in the ssh-action will be executed by oozie user, then you will need to set oozie as a bash user.

To do that you need to change the /etc/passwd file on all the nodes of the cluster. Look for the below value (similar to it) in the /etc/passwd file.

oozie:x:488:487:Oozie User:/var/lib/oozie:/bin/false 

and change it to

oozie:x:488:487:Oozie User:/var/lib/oozie:/bin/bash

which will actually make oozie user a bash user. And then proceed with the password-less authentication between the oozie user and any other user that you want on any of the host machine.

And then try to rerun the oozie job again. And let me know if it works. Hope it helps!!!

Answer 4

This is a very tricky problem and I could only hack it. I wasnt satisfied with the answer given so here my my version. Following failed for me( I could see in logs )

ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@XXX.XX.XX.XXX mkdir -p oozie-oozi/0000067-130808155814753-oozie-oozi-W/mysshjob--ssh/

But if tried the same command but removed KbdInteractiveDevices=no or changed KbdInteractiveDevices=pam it worked

ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=pam -o StrictHostKeyChecking=no -o ConnectTimeout=20 user@XXX.XX.XX.XXX mkdir -p oozie-oozi/0000067-130808155814753-oozie-oozi-W/mysshjob--ssh/

Anyway I think there was some issue with old ssh key so I tried following and it works

$ ssh-keygen -t dsa
$ cat ~/.ssh/id_dsa.pub > ~/.ssh/authorized_keys2