gnu-parallel

问题 I have list of shell commands that I'd like to call. Up to four processes shall run at the same time. As soon as the process count drops below 4, the next command is sent to the shell until all command have finished. I already got a "partly" working solution from stackoverflow fellows that however rely on wait -n which is not available on my debian wheezy installation. Link to other stackoverflow question One guy mentioned GNU Parallel. Can this be achieved? Any guidance is really appreciated

问题 Has anyone been able to successfully use GNU Parallel on Windows 10 with git-bash? Is it possible? - If so, how? Background: I'm having trouble installing GNU Parallel and using it, and it got me thinking - maybe git-bash is holding me back? I'm sure if I installed Ubuntu through WSL I wouldn't have any problems running GNU Parallel. But I wanted to know if I could do this in git-bash first. 回答1: I just installed git-bash on a Microsoft Windows 10 machine and had no problems installing GNU

问题 Has anyone been able to successfully use GNU Parallel on Windows 10 with git-bash? Is it possible? - If so, how? Background: I'm having trouble installing GNU Parallel and using it, and it got me thinking - maybe git-bash is holding me back? I'm sure if I installed Ubuntu through WSL I wouldn't have any problems running GNU Parallel. But I wanted to know if I could do this in git-bash first. 回答1: I just installed git-bash on a Microsoft Windows 10 machine and had no problems installing GNU

问题 I am trying to run a command to download 3000 files in parallel. I am using Cygwin + Windows. Downloading a single file via WGET in terminal : wget --no-check-certificate --content-disposition --load-cookies cookies.txt \ -p https://username:password@website.webpage.com/folder/document/download/1?type=file allows me to download the file with ID 1 singularly, in the correct format (as long as --content-disposition is in the command). I iterate over this REST API call to download the entire

问题 I would like to use gnu-parallel to parallelize the following snippet for a in `seq 5` ; do for b in `seq $a` ; do echo $a $b done done Is this possible to achieve using something like parallel echo {1} {2} ::: $(seq 5) ::: $(seq {1}) The above syntax does not work. Motivation: I would like to use parallel in order to submit a number of jobs to a cluster, as explained here http://docs.rcc.uchicago.edu/software/scheduler/parallel/README.html 回答1: GNU Parallel can do the cartesian product. What

问题 Running the following command does what I want when reading from a file: parallel --gnu -j2 "echo {} && sleep 5" < myfile.txt I would like to do something similar with a pipe. Note that I used the following page for inspiration for the pipe reader and writer: http://www.linuxjournal.com/content/using-named-pipes-fifos-bash Here is my pipe reader file: #!/bin/bash pipe=/tmp/testpipe trap "rm -f $pipe" EXIT if [[ ! -p $pipe ]]; then mkfifo $pipe fi while true do parallel --gnu -j2 "echo {} &&

问题 Imagine that you use gnu parallel to run several script that load every time a big object in memory, like a corpus of document for example. In the case of a read only acces to this object the question is: Can I load this object once in memory and share the read access by all the process running under Gnu Parallel ? 来源： https://stackoverflow.com/questions/34933830/would-it-be-possible-to-share-some-memory-with-gnu-parallel

问题 I am trying to install a set of sub-directories from the parent dir using GNU parallel . I'd like to run certain commands for all directories. Installing ls -d -- */ | grep -v 'node_modules' | parallel "npm i" Removing node_modules ls -d -- */ | grep -v 'node_modules' | parallel "rm -rf node_modules" Create README.md ls -d -- */ | grep -v 'node_modules' | parallel "touch README.md" How can I fire the first argument of parallel as a command within each directory passed to parallel ? 回答1: ls -d

问题 EDIT 2 After trying to run parallel, I am getting the following on the terminal when I run both parallel and by the source /homedtic/gsantamaria/parallel/bin/parallel : perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = "en_US:en", LC_ALL = (unset), LC_PAPER = "en_GB.UTF-8", LC_ADDRESS = "en_GB.UTF-8", LC_MONETARY = "en_GB.UTF-8", LC_NUMERIC = "en_GB.UTF-8", LC_TELEPHONE = "en_GB.UTF-8", LC_IDENTIFICATION = "en_GB.UTF-8", LC_MEASUREMENT =

问题 I'm running a script.sh in a loop. The script contains a parallel wget command. I'm getting the following error: Signal SIGCHLD received, but no signal handler set. The loop looks like this: for i in {1..5}; do /script.sh; done And the line that is causing the error looks like this (omitting options and settings): cat file.txt | parallel -j15 wget Research: I'm not an expert with GNU Parallel, but the script seems to work fine most of the time except when I get the error above. While looking