qsub

I am trying to adjust some bash scripts to make them run on a ( pbs ) cluster. The individual tasks are performed by several script thats are started by a main script. So far this main scripts starts multiple scripts in background (by appending & ) making them run in parallel on one multi core machine. I want to substitute these calls by qsub s to distribute load accross the cluster nodes. However, some jobs depend on others to be finished before they can start. So far, this was achieved by wait statements in the main script. But what is the best way to do this using the grid engine? I already

I have a qsub script as #####----submit_job.sh---##### #!/bin/sh #$ -N job1 #$ -t 1-100 #$ -cwd SEEDFILE=/home/user1/data1 SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE) /home/user1/run.sh $SEED The problem is-- it puts all error and output files (job1.eJOBID & job1.oJOBID) to the same directory from where I am running qsub submit_job.sh while I want to save these file (output and error log file in same different place (specified as $SEED_output). I tried to change the line as /home/user1/run.sh $SEED -o $SEED_output But it didn't work. Any suggestion ?? How can I specify path and name of

This question is related to pbs job no output when busy . i.e Some of the jobs I submit produce no output when PBS/Torque is 'busy'. I imagine that it is busier when many jobs are being submitted one after another, and as it so happens, of the jobs submitted in this fashion, I often get some that do not produce any output. Here're some codes. Suppose I have a python script called "x_analyse.py" that takes as its input a file containing some data, and analyses the data stored in the file: ./x_analyse.py data_1.pkl Now, suppose I need to: (1) Prepare N such data files: data_1.pkl, data_2.pkl, ..

I'm having an issue passing variables to a Bash script using QSub. Assume I have a Bash script named example. The format of example is the following: #!/bin/bash # (assume other variables have been set) echo $1 $2 $3 $4 So, executing "bash example.sh this is a test" on Terminal (I am using Ubuntu 12.04.3 LTS, if that helps) produces the output "this is a test". However, when I enter "qsub -v this,is,a,test example.sh", I get no output. I checked the output file that QSub produces, but the line "this is a test" is nowhere to be found. Any help would be appreciated. Thank you. Using PBSPro or

how can I parametrize a shell script that is executed on a grid (started with qsub) ? I have a shell script, where I use getopts to read the parameters. When I start (qsub script.sh -r firstparam -s secondparam ..) this working script with qsub I receive error messages, qsub: invalid option -- s qsub: illegal -r value as qsub thinks the parameter are for itself. Yet I have not found any solution. Thanks Using the qsub -v option is the proper way: qsub -v par_name=par_value[,par_name=par_value...] script.sh par_name can be used as variable in the shell script. jhrf In addition to volk's answer,

I have a batch script which starts off a couple of qsub jobs, and I want to trap when they are all completed. I don't want to use the -sync option, because I want them to be running simultaneously. Each job has a different set of command line parameters. I want my script to wait till when all the jobs have been completed, and do something after that. I don't want to use the sleep function e.g. to check if certain files have been generated after each 30 s, because this is a drain on resources. I believe Torque may have some options, but I am running SGE. Any ideas on how I could implement this