ansible - delete unmanaged files from directory?

Question

I want to recursively copy over a directory and render all .j2 files in there as templates. For this I am currently using the following lines:

Answer 1

We do this with our nginx files, since we want them to be in a special order, come from templates, but remove unmanaged ones this works:

  # loop through the nginx sites array and create a conf for each file in order
  # file will be name 01_file.conf, 02_file.conf etc
  - name: nginx_sites conf
    template: >
      src=templates/nginx/{{ item.1.template }}
      dest={{ nginx_conf_dir }}/{{ '%02d' % item.0 }}_{{ item.1.conf_name|default(item.1.template) }}
      owner={{ user }}
      group={{ group }}
      mode=0660
    with_indexed_items: nginx_sites
    notify:
      - restart nginx
    register: nginx_sites_confs

  # flatten and map the results into simple list
  # unchanged files have attribute dest, changed have attribute path
  - set_fact:
      nginx_confs: "{{ nginx_sites_confs.results|selectattr('dest', 'string')|map(attribute='dest')|list + nginx_sites_confs.results|selectattr('path', 'string')|map(attribute='path')|select|list }}"
    when: nginx_sites

  # get contents of conf dir
  - shell: ls -1 {{ nginx_conf_dir }}/*.conf
    register: contents
    when: nginx_sites

  # so we can delete the ones we don't manage
  - name: empty old confs
    file: path="{{ item }}" state=absent
    with_items: contents.stdout_lines
    when: nginx_sites and item not in nginx_confs

The trick (as you can see) is that template and with_items have different attributes in the register results. Then you turn them into a list of files you manage and then get a list of the the directory and removed the ones not in that list.

Could be done with less code if you already have a list of files. But in this case I'm creating an indexed list so need to create the list as well with map.

Answer 2

Here's something I came up with:

- template: src=/source/directory{{ item }}.j2 dest=/target/directory/{{ item }}
  register: template_results
  with_items:
    - a_list.txt
    - of_all.txt
    - templates.txt
- set_fact:
    managed_files: "{{ template_results.results|selectattr('invocation', 'defined')|map(attribute='invocation.module_args.dest')|list }}"

- debug:
    var: managed_files
    verbosity: 0

- find:
    paths: "/target/directory/"
    patterns: "*.txt"
  register: all_files
- set_fact:
    files_to_delete: "{{ all_files.files|map(attribute='path')|difference(managed_files) }}"

- debug:
    var: all_files
    verbosity: 0
- debug:
    var: files_to_delete
    verbosity: 0

- file: path={{ item }} state=absent
  with_items: "{{ files_to_delete }}"

• This generates the templates (however which way you want), and records the results in 'template_results'
• The the results are mangled to get a simple list of the "dest" of each template. Skipped templates (due to a when condition, not shown) have no "invocation" attribute, so they're filtered out.
• "find" is then used to get a list of all files that should be absent unless specifically written.
• this is then mangled to get a raw list of files present, and then the "supposed to be there" files are removed.
• The remaining "files_to_delete" are then removed.

Pros: You avoid multiple 'skipped' entries showing up during deletes.

Cons: You'll need to concatenate each template_results.results if you want to do multiple template tasks before doing the find/delete.

Answer 3

Usually I do not remove files but I add -unmanaged suffix to its name. Sample ansible tasks:

- name: Get sources.list.d files
  shell: grep -r --include=\*.list -L '^# Ansible' /etc/apt/sources.list.d || true
  register: grep_unmanaged
  changed_when: grep_unmanaged.stdout_lines

- name: Add '-unmanaged' suffix
  shell: rename 's/$/-unmanaged/' {{ item }}
  with_items: grep_unmanaged.stdout_lines

EXPLANATION

Grep command uses:

• -r to do recursive search
• --include=\*.list - only take files with .list extension during recursive search
• -L '^# Ansible' - display file names that are not having line starting with '# Ansible'
• || true - this is used to ignore errors. Ansible's ignore_errors also works but before ignoring the error ansible will show it in red color during ansible-playbook run which is undesired (at least for me).

Then I register output of grep command as a variable. When grep displays any output I set this task as changed (the line changed_when is responsible for this).

In next task I iterate grep output (i.e. file names returned by grep) and run rename command to add suffix to each file.

That's all. Next time you run the command first task should be green and second skipped.

Answer 4

There might be a couple of ways to handle this, but would it be possible to entirely empty the target directory in a task before the template step? Or maybe drop the templated files into a temporary directory and then delete+rename in a subsequent step?

Answer 5

Apparently this isn't possible with ansible at the moment. I had a conversation with mdehaan on IRC and it boils down to ansible not having a directed acyclic graph for resources, making things like this very hard.

Asking mdehaan for an example e.g. authoritatively managing a sudoers.d directory he came up with these things:

14:17 < mdehaan> Robe: http://pastebin.com/yrdCZB0y
14:19 < Robe> mdehaan: HM
14:19 < Robe> mdehaan: that actually looks relatively sane
14:19 < mdehaan> thanks :)
14:19 < Robe> the problem I'm seeing is that I'd have to gather the managed files myself
14:19 < mdehaan> you would yes
14:19 < mdehaan> ALMOST
14:20 < mdehaan> you could do a fileglob and ... well, it would be a little gross
[..]
14:32 < mdehaan> eh, theoretical syntax, nm
14:33 < mdehaan> I could do it by writing a lookup plugin that filtered a list
14:34 < mdehaan> http://pastebin.com/rjF7QR24
14:34 < mdehaan> if that plugin existed, for instance, and iterated across lists in A that were also in B

Answer 6

I'd do it like this, assuming a variable defined as 'managed_files' up top that is a list.

- shell: ls -1 /some/dir
  register: contents

- file: path=/some/dir/{{ item }} state=absent
  with_items: contents.stdout_lines
  when: item not in managed_files