process-substitution

Why does `script.py <(cat *.gz)` work with subprocess.Popen in python 2 but not python 3?

一世执手 提交于 2021-01-29 03:20:48
问题 We discovered recently that a script we developed chokes in python 3.x (but not python 2.x) if it is supplied its input files via process substitution, e.g.: script.py <(cat *.gz) We've tested with commands other than gzip, such as cat, just to see if we get a similar error. They all complain that /dev/fd/63 (or /dev/fd/63.gz ) does not exist. Here's the (simplified) relevant bit of code: def open_gzip_in(infile): '''Opens a gzip file for reading, using external gzip if available''' #

Bash process substitution behaves differently than named pipe?

房东的猫 提交于 2020-06-16 16:58:47
问题 Kind of a follow up to this question. I use bash process substitution expecting to see 5 lines of output (corresponding to 5s timeout) however I only see 4 lines: timeout 5s dd if=/dev/random | pv -fr > /dev/null 2> >(tr '\r' '\n' >&2) [5.42kiB/s] [1.89kiB/s] [5.36kiB/s] [2.41kiB/s] However if I place a named pipe in the middle I do get the 5 lines as expected. First shell: mkfifo testfifo cat testfifo | tr '\r' '\n' >&2 Second shell: timeout 5s dd if=/dev/random | pv -fr > /dev/null 2>

Bash: What is the scope of the process substitution?

浪子不回头ぞ 提交于 2020-06-14 08:46:30
问题 As far as I know, process substitution <(...) / >(...) creates the fd and stores the output of commands in parentheses into the generated fd. Therefore, these two commands are equivalent $ ls -al $ cat <(ls -al) Here, my question is, how long the generated file descriptors remain? I've read this article, but seems my understanding is wrong. If a process substitution is expanded as an argument to a function, expanded to an environment variable during calling of a function, or expanded to any

Bash: What is the scope of the process substitution?

回眸只為那壹抹淺笑 提交于 2020-06-14 08:46:29
问题 As far as I know, process substitution <(...) / >(...) creates the fd and stores the output of commands in parentheses into the generated fd. Therefore, these two commands are equivalent $ ls -al $ cat <(ls -al) Here, my question is, how long the generated file descriptors remain? I've read this article, but seems my understanding is wrong. If a process substitution is expanded as an argument to a function, expanded to an environment variable during calling of a function, or expanded to any

Bash: What is the scope of the process substitution?

假装没事ソ 提交于 2020-06-14 08:45:18
问题 As far as I know, process substitution <(...) / >(...) creates the fd and stores the output of commands in parentheses into the generated fd. Therefore, these two commands are equivalent $ ls -al $ cat <(ls -al) Here, my question is, how long the generated file descriptors remain? I've read this article, but seems my understanding is wrong. If a process substitution is expanded as an argument to a function, expanded to an environment variable during calling of a function, or expanded to any

Is skipping/ignoring NUL bytes on process substitution standardized?

余生长醉 提交于 2020-01-25 01:48:18
问题 Executive Summary Is it standard behavior that shells skip over NUL bytes when doing process substitution? For example, executing printf '\0abc' | read value && echo $value will yield abc . The NUL value is skipped, even though the hexdump of the printf output shows it's clearly being output. My first thought was " word splitting ". However, when using an actual process substitution value=$(printf '\0abc') the results are similar and = does not perform word splitting. Long Story While

bash: How do I ensure termination of process substitution used with exec?

我是研究僧i 提交于 2020-01-21 05:23:05
问题 If I run $#/bin/bash for i in `seq 5`; do exec 3> >(sed -e "s/^/$i: /"; echo "$i-") echo foo >&3 echo bar >&3 exec 3>&- done then the result is not synchronous; it could be something like: 1: foo 1: bar 2: foo 2: bar 1- 3: foo 3: bar 2- 3- 4: foo 5: foo 4: bar 5: bar 4- 5- How do I ensure that the process substitution >(...) is completed before proceeding to the next iteration? Inserting sleep 0.1 after exec 3>&- helped, but it's inelegant, inefficient, and not guaranteed to always work. EDIT

Why does `cat <(cat)` produce EIO?

杀马特。学长 韩版系。学妹 提交于 2020-01-01 08:38:09
问题 I have a program that reads from two input files simultaneously. I'd like to have this program read from standard input. I thought I'd use something like this: $program1 <(cat) <($program2) but I've just discovered that cat <(cat) produces .... mmap2(NULL, 139264, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb758e000 read(0, 0xb758f000, 131072) = -1 EIO (Input/output error) .... cat: -: Input/output error and similarly, $ cat <(read -n 1) bash: read: read error: 0: Input/output

Do some programs not accept process substitution for input files?

大憨熊 提交于 2019-12-30 09:44:15
问题 I'm trying to use process substitution for an input file to a program, and it isn't working. Is it because some programs don't allow process substitution for input files? The following doesn't work: bash -c "cat meaningless_name" >sequence1 gattacagattacagattacagattacagattacagattacagattacagattaca >sequence2 gattacagattacagattacagattacagattacagattacagattacagattaca bash -c "clustalw -align -infile=<(cat meaningless_name) -outfile=output_alignment.aln -newtree=output_tree.dnd" (Less verbose

Bash script process substitution Syntax error: “(” unexpected

白昼怎懂夜的黑 提交于 2019-12-28 02:19:05
问题 I want to run this script: #!/bin/bash echo <(true) I run it as: sh file.sh And I get " Syntax error: "(" unexpected " . I found some similar situations but still can't solve this. I'm a beginner at shell scripting , but as I understand: the shebang I use is correct and chooses the bash shell , so the process substitution syntax should work I try the same from the command line and it works. I checked with echo $0 and it gives me " bash " , so what's the difference from running the command in