GNU parallel --jobs option using multiple nodes on cluster with multiple cpus per node

后端 未结 2 1443
一向
一向 2020-12-31 18:24

I am using gnu parallel to launch code on a high performance (HPC) computing cluster that has 2 CPUs per node. The cluster uses TORQUE portable batch system (PBS). My questi

2条回答
  •  囚心锁ツ
    2020-12-31 19:01

    1. Yes: -j is the number of jobs per node.
    2. Yes: Install 'parallel' in your $PATH on the remote hosts.
    3. Yes: It is a consequence from parallel missing from the $PATH.

    GNU Parallel logs into the remote machine; tries to determine the number of cores (using parallel --number-of-cores) which fails and then defaults to 1 CPU core per host. By giving -j2 GNU Parallel will not try to determine the number of cores.

    Did you know that you can also give the number of cores in the --sshlogin as: 4/myserver ? This is useful if you have a mix of machines with different number of cores.

提交回复
热议问题