Install data directory tree with massive number of files using automake

Question

I have a data directory which I would like automake to generate install and uninstall targets for. Essentially, I just want to copy this directory verbatim to the DATA director

Answer 1

I would use a script to generate a Makefile fragment that lists all the files:

echo 'subdir_files =' > subfiles.mk
find subdir -type f -print | sed 's/^/  /;$q;s/$/ \\/' >> subfiles.mk

and then include this subfiles.mk from your main Makefile.am:

include $(srcdir)/subfiles.mk
nobase_dist_pkgdata_DATA = $(subdir_files)

---

A second option is to EXTRA_DIST = subdir, and then to write custom install-data-local and uninstall-local rules.

The problem here is that EXTRA_DIST = subdir will distributes all files in subdir/, including backup files, configuration files (e.g. from your VCS), and other things you would not want to distribute.

Using a script as above let you filter the files you really want to distribute.

Answer 2

I've found that installing hundreds of files separately makes for a tormentingly long invocation of make install. I had a similar case where I wanted to install hundreds of files, preserving the directory structure, and I did not want to change my Makefile.am every time a file was added to or removed from the collection.

I included a LZMA archive of the files in my distribution, and made automake rules like so:

GIANTARCHIVE = My_big_archive.tar.lz

dist_pkgdata_DATA = $(GIANTARCHIVE)

install-data-hook:
    cd $(DESTDIR)$(pkgdatadir); \
    cat $(GIANTARCHIVE) | unlzma | $(TAR) --list > uninstall_manifest.txt; \
    cat $(GIANTARCHIVE) | unlzma | $(TAR) --no-same-owner --extract; \
    rm --force $(GIANTARCHIVE); \
    cat uninstall_manifest.txt | sed --expression='s/^\|$$/"/g' | xargs chmod a=rX,u+w

uninstall-local:
    cd $(DESTDIR)$(pkgdatadir); \
    cat uninstall_manifest.txt | sed --expression='s/ /\\ /g' | xargs rm --force; \
    rm --force uninstall_manifest.txt

This way, automake installs My_big_archive.tar.lz in the $(pkgdata) directory, and extracts it there, making a list of all the files that were in it, so it can uninstall them later. This also runs much faster than listing each file as an install target, even if you were to autogenerate that list.

Answer 3

I would write a script (either as a separate shell script, or in the Makefile.am), that is run as part of the install-data-hook target.