sungridengine

Timeout jobs on sun grid engine

由 落花浮王杯 提交于 2019-12-11 07:02:02
问题 I'm running a lot of jobs with sun grid engine (linux). Some of the jobs take a (very) long time to run, and I don't know ahead which ones. I would like to stop jobs that run for more than, say, 2 hours. Is it possible to run using SGE? Is it possible to do it from the unix shell? Thanks 回答1: If you're running the jobs yourself then use the hard wall clock time. #$ -l h_rt=2:00:00 Where time is hr:min:sec 来源： https://stackoverflow.com/questions/3587053/timeout-jobs-on-sun-grid-engine

Redirect output of my java program under qsub

由 杀马特。学长 韩版系。学妹 提交于 2019-12-11 06:05:49
问题 I am currently running multiple Java executable program using qsub. I wrote two scripts: 1) qsub.sh, 2) run.sh qsub.sh #! /bin/bash echo cd `pwd` \; "$@" | qsub run.sh #! /bin/bash for param in 1 2 3 do ./qsub.sh java -jar myProgram.jar -param ${param} done Given the two scripts above, I submit jobs by sh run.sh I want to redirect the messages generated by myProgram.jar -param ${param} So in run.sh , I replaced the 4th line with the following ./qsub.sh java -jar myProgram.jar -param ${param}

Grid engine cluster + OpenCV: strange behaviour

由 不想你离开。 提交于 2019-12-11 04:56:45
问题 I'm using a Grid Engine cluster for running some OpenCV code. The code runs well when executed locally, but when submitted to the grid it's not working. I extracted here a minimal example. In the directory ~/code/ I have a file test.cpp containing the following code: #include <opencv2/core.hpp> #include <iterator> #include <string> #include <sys/types.h> #include <sys/stat.h> using namespace cv; using namespace std; int main(int ac, char** av) { /// Create a random matrix Mat M; /// Create a

efficient way to wait for job completion : python and drmaa

由 回眸只為那壹抹淺笑 提交于 2019-12-10 11:47:11
问题 I wanted to ask about "wait" feature in drmaa API I am using through Python. Does it do constant qstat's ( if we are running it on SGE) to check whether a program has finished execution. Our admin want us to avoid any constant qstat's as it slows down the performance due to extra load on scheduler. In general wat would be an efficient way to check for job status through DRMAA without overboarding the scheduler. Thanks! -Abhi 回答1: From an DRMAA API point of view, there is no better way. The

Syntax for submitting a qsub job without an actual job file?

由 孤者浪人 提交于 2019-12-08 01:32:05
问题 I would like to submit qsub jobs on the fly without creating discrete job files. So, let's say I have a python script called "get_time.py" that simply reports the time. Instead of making a submission script like this: cat>job.sub<<eof #PBS -l walltime=1:00:00 cd $PBS_O_WORKDIR get_time.py eof ...and then submitting the job: qsub job.sub I would like to be able to bypass the file creation step, and I'd image the construct would be something like this: qsub -d . -e get_time.py where -e is my

SunGridEngine, Condor, Torque as Resource Managers for PVM

由 你说的曾经没有我的故事 提交于 2019-12-07 02:36:55
问题 Anyone have any idea which Resource manager is good for PVM? Or should I not have used PVM and instead relied on MPI (or any version of it, such as MPICH-2 [are there any other ones that are better?]). Main reason for using PVM was because the person before me who started this project assumed the use of PVM. However, now that this project is mine (he hasn't done any significant work that relies on PVM) this can be easily changed, preferably to something that is easy to install because

Syntax for submitting a qsub job without an actual job file?

由 吃可爱长大的小学妹 提交于 2019-12-06 11:47:20
I would like to submit qsub jobs on the fly without creating discrete job files. So, let's say I have a python script called "get_time.py" that simply reports the time. Instead of making a submission script like this: cat>job.sub<<eof #PBS -l walltime=1:00:00 cd $PBS_O_WORKDIR get_time.py eof ...and then submitting the job: qsub job.sub I would like to be able to bypass the file creation step, and I'd image the construct would be something like this: qsub -d . -e get_time.py where -e is my imaginary parameter that tells qsub that the following is code to be sent to the scheduler, instead of

how to specify error log file and output file in qsub

由 我是研究僧i 提交于 2019-12-05 12:02:52
问题 I have a qsub script as #####----submit_job.sh---##### #!/bin/sh #$ -N job1 #$ -t 1-100 #$ -cwd SEEDFILE=/home/user1/data1 SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE) /home/user1/run.sh $SEED The problem is-- it puts all error and output files (job1.eJOBID & job1.oJOBID) to the same directory from where I am running qsub submit_job.sh while I want to save these file (output and error log file in same different place (specified as $SEED_output). I tried to change the line as /home/user1/run

SunGridEngine, Condor, Torque as Resource Managers for PVM

由 半世苍凉 提交于 2019-12-05 06:07:40
Anyone have any idea which Resource manager is good for PVM? Or should I not have used PVM and instead relied on MPI (or any version of it, such as MPICH-2 [are there any other ones that are better?]). Main reason for using PVM was because the person before me who started this project assumed the use of PVM. However, now that this project is mine (he hasn't done any significant work that relies on PVM) this can be easily changed, preferably to something that is easy to install because installing and setting up PVM was a big hassle. I'm leaning towards SunGridEngine seeing as how I have

SnakeMake rule with Python script, conda and cluster

由 萝らか妹 提交于 2019-12-04 15:21:18
I would like to get snakemake running a Python script with a specific conda environment via a SGE cluster. On the cluster I have miniconda installed in my home directory. My home directory is mounted via NFS so accessible to all cluster nodes. Because miniconda is in my home directory, the conda command is not on the operating system path by default. I.e., to use conda I need to first explicitly add this to the path. I have a conda environment specification as a yaml file, which could be used with the --use-conda option. Will this work with the --cluster "qsub" option also? FWIW I also launch
订阅 sungridengine