Executing an R script in python via subprocess.Popen

Question

When I execute the script in R, it is:

$ R --vanilla --args test_matrix.csv < hierarchical_clustering.R > out.txt

In Python, it works if

Answer 1

A couple of ideas:

1. You might want to consider using the Rscript frontend, which makes running scripts easier; you can pass the script filename directly as a parameter, and do not need to read the script in through standard input.
2. You don't need the shell for just redirecting standard output to a file, you can do that directly with subprocess.Popen.

Example:

import subprocess

output_name = 'something'
script_filename = 'hierarchical_clustering.R'
param_filename = '%s_DM_Instances_R.csv' % output_name
result_filename = '%s_out.txt' % output_name
with open(result_filename, 'wb') as result:
    process = subprocess.Popen(['Rscript', script_filename, param_filename],
                               stdout=result);
    process.wait()

Answer 2

You never actually execute it fully ^^ try the following

process = subprocess.Popen(['R', '--vanilla', '--args', '\\%s_DM_Instances_R.csv\\' % output_filename, '<', '/home/kevin/AV-labels/Results/R/hierarchical_clustering.R'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=True) 
process.communicate()#[0] is stdout

Answer 3

I've solved this problem by putting everything into the brackets..

process = subprocess.Popen(["R --vanilla --args "+output_filename+"_DM_Instances_R.csv < /home/kevin/AV-labels/Results/R/hierarchical_clustering.R > "+output_filename+"_out.txt"], shell=True)
process.wait()

Answer 4

Keven's solution works for my requirement. Just to give another example about @Kevin's solution. You can pass more parameters to the rscript with python-style string:

import subprocess

process = subprocess.Popen(["R --vanilla --args %s %d %.2f < /path/to/your/rscript/transformMatrixToSparseMatrix.R" % ("sparse", 11, 0.98) ], shell=True)
process.wait()

Also, to make things easier you could create an R executable file. For this you just need to add this in the first line of the script:

#! /usr/bin/Rscript --vanilla --default-packages=utils

Reference: Using R as a scripting language with Rscript or this link

Answer 5

Instead of 'R', give it the path to Rscript. I had the same problem. Opens up R but doesn't execute my script. You need to call Rscript (instead of R) to actually execute the script.

retcode = subprocess.call("/Pathto/Rscript --vanilla /Pathto/test.R", shell=True)

This works for me.

Cheers!