Python Script not running in crontab calling pysaunter

Question

I have read multiple posts and many articles detailing that scipts in a cron job need to keep the environment variables necessary to run inside the script itself due to the

Answer 1

After much trial and error, and many, many stackoverflow.com articles and other tutorials online, and with the help of a Perl script I found that did something similar, I was able to figure out what needed to be done to get this to work.

Here are the steps to making sure everything is set up correctly:

1. Make sure that you have the variables you need in PYTHONPATH (found here and here, and for more info. go here) inside the .profile or .bash_profile for any shell you want to test your script in to make sure it works.

2. Edit your crontab to include the directories needed to run your script in a cron job (found here and here).

   a) Be sure to include the root directory in the PATH variable (.) as explained here. Basically, if you are running an executable with your command it needs to be able to find root or the directory where the executable is stored and also probably these: (/sbin:/bin:/usr/sbin:/usr/bin).

3. In your crontab file, create a cronjob that will change the current directory to the directory where you have successfully run the script before (e.g., /Users/user/Documents/foo).

   a) This will look like the following:

   * * * * cd /Users/user/Documents/foo; bar -l doSomething -v 
   

4. Since my problem dealt specifically with calling an executable, it is necessary to note that there are multiple ways of writing the Python script to run (though in the process of discovery I learned that this works for any call made using subprocess in cron).

   The first method looks like this:

   ... #some script calls
   my_env = os.environ.copy()
   my_env["PYTHONPATH"] = "{}:{}".format(os.environ["PATH"] ,"<path you want to include>")
   os.chdir("<path/to/desired/directory>")
   subprocess.Popen(<call_as_string>, env=my_env, shell=True)
   

   And the second looks like this:

   ... #some script calls
   os.environ["PYTHONPATH"] = "{}:{}".format(os.environ["PATH"] ,"<path you want to include>")
   os.chdir("<path/to/desired/directory>")
   subprocess.Popen(<call_as_list_of_arguments)
   

   Since the executable needed the path to the helpers directory included in the shell where it was called from, it was necessary to pass the environment variable to the executable as explained here. What I found, though, is that modifying the PATH variable in the environment didn't work for a cron job, but setting the PYTHONPATH did. I read here that the PATH variable is used by the shell to only look for executables, so for a new shell inside a cronjob you need to pass the shell a PYTHONPATH to look for new Python modules. (This is also explained in the Python docs.)

   The differences between the two different methods are explained in the subprocess documentation cited in the question, but a good tutorial on this module can be found here.

Answer 2

What you've said makes no sense.

You say shell_env = sys.path. sys.path is a list of folders for python to look for modules, not an environment variable mapping!

You then use that in subprocess.call. Perhaps you meant to write env=my_env.

That's the next problem. First, PATH should be a list of folder separated by ':'. You just tack shell_path onto the last folder.

Finally, python uses PYTHONPATH as a list to locate python modules (which seems to be what problem you're having.)