Working from raw source data from a filepath containing spaces using make and Makefiles

放肆的年华 提交于 2021-01-29 09:43:34

问题


I have a repository that uses python scripts and a Makefile. I want to have a setup procedure that allows them to easily set up an environment and copy in the necessary data files from our server.

The problem with including the source data files in the Makefile is that the company server uses spaces in the drive name, which make doesn't like very much, so I can list those files as dependencies for the target output file.

My current Makefile basically does only the following:

.PHONY : all
all : output.csv

.PHONY : copy_data_to_local_folder
copy_data_to_local_folder :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" local/inputs1.csv
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" local/inputs2.csv

output.csv : combine_data.R local/inputs1.csv local/inputs2.csv
    Rscript $^ $@

The copy_data_to_local_folder part is just to get the data to the local directory, but it isn't included in the DAG leading to the production of output.csv (i.e. all : output.csv copy_data_to_local_folder) or else the target would need to run everytime.

My solution ideas are the following, but I'm not sure what's best practice:

  1. Use a different make tool. I could use Luigi in Python or Drake in R, but I would prefer to keep the tool somewhat more generalized.

  2. Run a setup script to copy in files. I assume there would be a way to run the file copying scripts as part of the environment setup, but I am unfamiliar with how to do this.

I am not sure about the best way to do this. I want to be able to share the code with a co-worker and have them be able to get up and running on their system without too much messing around to configure. Is there a best practice for this situation?


回答1:


One fix would be:

local/inputs1.csv :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 1.csv" $@
local/inputs2.csv :
    python copyfile.py "V:\\Server Path\\With Spaces\\Inputs 2.csv" $@

output.csv : combine_data.R | local/inputs1.csv local/inputs2.csv
    Rscript $^ $| $@

Note that local/inputs1.csv and local/inputs2.csv are made order-only prerequisites, so that they are only made when they don't exist (unless you'd like to copy them every time the makefile is run). Automatic variable $| refers to order-only prerequisites, they aren't included in $^.



来源:https://stackoverflow.com/questions/60743710/working-from-raw-source-data-from-a-filepath-containing-spaces-using-make-and-ma

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!