问题
I have a repository that uses python scripts and a Makefile. I want to have a setup procedure that allows them to easily set up an environment and copy in the necessary data files from our server.
The problem with including the source data files in the Makefile is that the company server uses spaces in the drive name, which make doesn't like very much, so I can list those files as dependencies for the target output file.
My current Makefile basically does only the following:
.PHONY : all
all : output.csv
.PHONY : copy_data_to_local_folder
copy_data_to_local_folder :
python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" local/inputs1.csv
python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" local/inputs2.csv
output.csv : combine_data.R local/inputs1.csv local/inputs2.csv
Rscript $^ $@
The copy_data_to_local_folder
part is just to get the data to the local directory, but it isn't included
in the DAG leading to the production of output.csv
(i.e. all : output.csv copy_data_to_local_folder
) or else
the target would need to run everytime.
My solution ideas are the following, but I'm not sure what's best practice:
Use a different make tool. I could use
Luigi
in Python orDrake
in R, but I would prefer to keep the tool somewhat more generalized.Run a setup script to copy in files. I assume there would be a way to run the file copying scripts as part of the environment setup, but I am unfamiliar with how to do this.
I am not sure about the best way to do this. I want to be able to share the code with a co-worker and have them be able to get up and running on their system without too much messing around to configure. Is there a best practice for this situation?
回答1:
One fix would be:
local/inputs1.csv :
python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" $@
local/inputs2.csv :
python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" $@
output.csv : combine_data.R | local/inputs1.csv local/inputs2.csv
Rscript $^ $| $@
Note that local/inputs1.csv
and local/inputs2.csv
are made order-only prerequisites, so that they are only made when they don't exist (unless you'd like to copy them every time the makefile is run). Automatic variable $|
refers to order-only prerequisites, they aren't included in $^
.
来源:https://stackoverflow.com/questions/60743710/working-from-raw-source-data-from-a-filepath-containing-spaces-using-make-and-ma