Why is python no longer waiting for os.system to finish?

Question

I have the following function, which has been working great for months. I have not updated my version of Python (unless it happens behind the scenes?).

def Blast(type, protein_sequence, start, end, genomic_sequence):
    result = []
    M = re.search('M', protein_sequence)
    if M:
        query = protein_sequence[M.start():]
        temp = open("temp.ORF", "w")
        print >>temp, '>blasting'
        print >>temp, query
        temp.close()
        cline = blastp(query="'temp.ORF'", db="DB.blast.txt",
                       evalue=0.01, outfmt=5, out=type + ".BLAST")
        os.system(str(cline))
        blast_out = open(type + ".BLAST")
        string = str(blast_out.read())
        DEF = re.search("<Hit_def>((E|L)\d)</Hit_def>", string)

I receive the error that blast_out=open(type+".BLAST") cannot find the specified file. This file gets created as part of the output of the program called by the os.system call. This usually takes ~30s or so to complete. However, When I try to run the program, it instantly gives the error I mention above.

I thought os.system() was supposed to wait for completion?
Should I force the wait somehow? (I do not want to hard code the wait time).

EDIT: I have ran the cline output in the command line version of the BLAST program. Everything appears to be fine.

Answer 1

os.system does wait. But there could be an error in the program called by it, so the file isn't created. You should check the return value of the called program before proceeding. In general, programs are supposed to return 0 when they finish normally, and another value when there is an error:

if os.system(str(cline)):
    raise RuntimeError('program {} failed!'.format(str(cline)))
blast_out=open(type+".BLAST")

Instead of raising an exception, you could also return from the Blast function, or try to handle it in another way.

Update: Wether the called program runs fine from the command line only tells you that there is nothing wrong with the program itself. Does the blast program return useful errors or messages when there is a problem? If so, consider using subprocess.Popen() instead of os.system, and capture the standard output as well:

prog = subprocess.Popen(cline, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = prog.communicate()
# Now you can use `prog.returncode`, and inspect the `out` and `err` 
# strings to check for things that went wrong.

Answer 2

You could also replace the call to os.system with subprocess.check_call, and that will raise an exception if the command fails:

import subprocess as subp
subp.check_call(str(cline), shell=True)

Answer 3

This answer is a bit late. However, I had the same problem and subprocess didn't seem to work. I solved it by writing the command into a bash-file and executing the bash-file via python os.system:

vi forPython.sh (write 'my command' into it)
chmod +x forPython.sh

(in Python script)

os.system("./forPython.sh")

This makes python wait for your process to finish.