Redirect stderr through grep -v in LSF batch job

问题

I'm using a library that generates a whole ton of output to stderr (and there is really no way to suppress the output directly in the code; it is ROOT's Minuit2 minimizer which is known for not having a way to suppress the output). I'm running batch jobs through the LSF system, and the error output files are so big that they exceed my disk quota. Erk.

When I run locally on a shell, I do:

python main.py 2> >( grep -v Minuit2 2>&1 )

to suppress the output, as is done here. This works great, but unfortunately I can't seem to get that or any variation of it to work when running on LSF. I think this is due to LSF not spawning the necessary subshell, but it's not clear.

I run on batch by passing LSF a submit script. The relevant line is:

python main.py $INPUT_FILE

which works great, aside from the aforementioned problem of gigantic error files.

When I try changing that line to

python main.py $INPUT_FILE 2> >( grep -v Minuit2 2>&1 )

I end up with

./singleSubmit.sh: line 16: syntax error near unexpected token `>'
./singleSubmit.sh: line 16: `python $MAIN $1 2> >( grep -v Minuit2 2>&1 )'

in the error log file.

Any idea how I could accomplish what I want, or why this is not working?

Thanks a ton!

回答1:

The syntax you're using works in bash, not in csh/tcsh. Try changing the first line of your submission script to

#!/bin/bash

来源：https://stackoverflow.com/questions/26707034/redirect-stderr-through-grep-v-in-lsf-batch-job