Use StringIO as stdin with Popen

Question

I have the following shell script that I would like to write in Python (of course grep . is actually a much more complex command):

#!/bin/bash

         


        

Answer 1

Don't use bare except, it may catch too much. In Python 3:

#!/usr/bin/env python3
from subprocess import check_output

try:
    file = open('somefile', 'rb', 0)
except FileNotFoundError:
    output = check_output(cmd, input=b'somefile not found')
else:
    with file:
        output = check_output(cmd, stdin=file)

It works for large files (the file is redirected at the file descriptor level -- no need to load it into the memory).

If you have a file-like object (without a real .fileno()); you could write to the pipe directly using .write() method:

#!/usr/bin/env python3
import io
from shutil import copyfileobj
from subprocess import Popen, PIPE
from threading import Thread

try:
    file = open('somefile', 'rb', 0)
except FileNotFoundError:
    file = io.BytesIO(b'somefile not found')

def write_input(source, sink):
    with source, sink:
        copyfileobj(source, sink)

cmd = ['grep', 'o']
with Popen(cmd, stdin=PIPE, stdout=PIPE) as process:
    Thread(target=write_input, args=(file, process.stdin), daemon=True).start()
    output = process.stdout.read()

Answer 2

p = subprocess.Popen(['grep', '...'], stdin=subprocess.PIPE, 
                                      stdout=subprocess.PIPE)
output, output_err = p.communicate(myfile.read())