Maven: how to filter the same resource multiple times with different property values?

Question

Our project uses Log4J, configured via log4j.properties file. We have multiple production servers, which log to different log files, so that the logs can be differentiated. So l

Answer 1

If there are a lot of target configurations that need to be copied, you could use the maven-antrun-plugin together with ant macrodef.

  <plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
      <execution>
        <id>copy-resources-all-nodes</id>
        <phase>process-resources</phase>
        <configuration>
          <tasks>
             <macrodef name="copyConfigFile">
                <attribute name="node"/>

                <sequential>
                   <copy file="src/main/resources/log4j.properties"
                         toFile="target/antrun/log4j-@{node}.properties">
                      <filterset>
                         <!-- put the node-specific config in property files node1.properties etc -->
                         <filtersfile file="config/@{node}.properties"/>
                      </filterset>
                   </copy>
                </sequential>
             </macrodef>

             <copyConfigFile node="node1"/>
             <copyConfigFile node="node2"/>
             <copyConfigFile node="node3"/>
             ...

          </tasks>
        </configuration>
        <goals>
          <goal>run</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

If you really have a lot of target configurations, you could also use ant-contrib to iterate over a list of target configurations.

There's an example how to do this here

Answer 2

Although this is a bit old, I came across the thread recently and would like to propose an updated solution using the iterator-maven-plugin. An overview is found here: http://khmarbaise.github.io/iterator-maven-plugin/

A specific example of how you can accomplish your goal would be to combine the iterator plugin with a copy resource and filter enabled. You could even add custom properties file to use as a filter in case you have other attributes that are unique per node with this approach.

<plugin>

    <groupId>com.soebes.maven.plugins</groupId>
    <artifactId>iterator-maven-plugin</artifactId>
    <version>0.3</version>

    <executions>
        <execution>
            <id>configure-log-properties</id>
            <phase>validate</phase>
            <goals>
                <goal>iterator</goal>
            </goals>
            <configuration>
                <iteratorName>log.file.postfix</iteratorName>
                <content>1,2,3,4,5</content>
                <pluginExecutors>

                    <pluginExecutor>
                        <plugin>
                            <artifactId>maven-resources-plugin</artifactId>
                            <version>2.7</version>
                        </plugin>

                        <goal>copy-resources</goal>

                        <configuration>
                            <outputDirectory>${project.build.directory}/nodes/${log.file.postfix}</outputDirectory>

                            <resources>
                                <resource>
                                    <directory>src/main/resources</directory>
                                    <includes>
                                        <include>log4j.properties</include>
                                    </includes>
                                    <filtering>true</filtering>
                                </resource>
                            </resources>
                        </configuration>

                    </pluginExecutor>
                </pluginExecutors>
            </configuration>
        </execution>
    </executions>
</plugin>

Answer 3

Here are some approaches that you can try:

1. use the antrun plugin, and the copy task to make duplicates of your resource file in the generate-resources phase. An example of using the antrun plugin is given in the answer to this SO question on copying with maven. You could even use ant's property expansion to expand each ${log.file.postfix} to a distinct value, either the literal values, 1,2,3 etc. or unique placeholders, ${log.file.postfix1}, ${log.file.postfix2} which are finally replaced when maven does the resource filtering.

2. Rather than using antrun, use your version control system to set up multiple copies of the same file. You can then run multiple instances of the resources:copy-resources goal, each with different property values configured, and a different target file name.

Answer 4

(...) I am fairly sure this can be done, e.g. via the antrun plugin, but I am not familiar with that. What is the simplest way to achieve this?

You could indeed use resources:copy-resources and several <execution> in your POM (note that resources:copy-resources doesn't allow to change the name of the target file though).

Let's assume you have the following structure:

$ tree .
.
├── pom.xml
└── src
    ├── main
    │   ├── filters
    │   │   ├── filter-node1.properties
    │   │   └── filter-node2.properties
    │   ├── java
    │   └── resources
    │       ├── log4j.properties
    │       └── another.xml
    └── test
        └── java

Where log4j.properties is using place holders and the filter-nodeN.properties files contain the values. For example:

# filter-node1.properties

log.location=D:/logs
log.file.postfix=_1

Then, in your pom.xml, configure the resources plugin and define one <execution> per node to call copy-resources with a specific output directory and a specific filter to use:

<project>
  ...
  <build>
    <resources>
      <!-- this is for "normal" resources processing -->
      <resource>
        <directory>src/main/resources</directory>
        <filtering>true</filtering><!-- you might still want to filter them -->
        <excludes>
          <!-- we exclude the file from "normal" resource processing -->
          <exclude>**/log4j.properties</exclude>
        </excludes>
      </resource>
    </resources>
    <plugins>
      <plugin>
        <artifactId>maven-resources-plugin</artifactId>
        <version>2.4.3</version>
        <executions>
          <execution>
            <id>copy-resources-node1</id>
            <phase>process-resources</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/node1</outputDirectory>
              <resources>
                <resource>
                  <directory>src/main/resources</directory>
                  <filtering>true</filtering>
                  <includes>
                    <include>**/log4j.properties</include>
                  </includes>
                </resource>
              </resources>
              <filters>
                <filter>src/main/filters/filter-node1.properties</filter>
              </filters>
            </configuration>
          </execution>
          <execution>
            <id>copy-resources-node2</id>
            <phase>process-resources</phase>
            <goals>
              <goal>copy-resources</goal>
            </goals>
            <configuration>
              <outputDirectory>${basedir}/target/node2</outputDirectory>
              <resources>
                <resource>
                  <directory>src/main/resources</directory>
                  <filtering>true</filtering>
                  <includes>
                    <include>**/log4j.properties</include>
                  </includes>
                </resource>
              </resources>
              <filters>
                <filter>src/main/filters/filter-node2.properties</filter>
              </filters>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Running mvn process-resources would produce the following result:

$ tree .
.
├── pom.xml
├── src
│   ├── main
│   │   ├── filters
│   │   │   ├── filter-node1.properties
│   │   │   └── filter-node2.properties
│   │   ├── java
│   │   └── resources
│   │       ├── log4j.properties
│   │       └── another.xml
│   └── test
│       └── java
└── target
    ├── classes
    │   └── another.xml
    ├── node1
    │   └── log4j.properties
    └── node2
        └── log4j.properties

With the appropriate values in each log4j.properties.

$ cat target/node1/log4j.properties 
log4j.appender.Application.File=D:/logs/application_1.log
log4j.appender.tx_info.File=D:/logs/tx_info_1.log

This kinda works, but is verbose and this might be a problem if you have a decent amount of nodes.

---

I tried to write something more concise and maintainable using the Maven AntRun Plugin but I couldn't get the for task from ant-contrib to work under Maven (for an unknown reason, the for task isn't recognized) and I gave up.

Here is an alternative using the Maven AntRun Plugin. Nothing complicated, no loop, I'm just copying the source file to another location, changing its name on the fly and filtering the content:

  <plugin>
    <artifactId>maven-antrun-plugin</artifactId>
    <version>1.3</version>
    <executions>
      <execution>
        <id>copy-resources-all-nodes</id>
        <phase>process-resources</phase>
        <configuration>
          <tasks>
            <copy file="src/main/resources/log4j.properties" toFile="target/antrun/log4j-node1.properties">
              <filterset>
                <filter token="log.location" value="D:/logs"/>
                <filter token="log.file.postfix" value="_1"/>
              </filterset>
            </copy>
            <copy file="src/main/resources/log4j.properties" toFile="target/antrun/log4j-node2.properties">
              <filterset>
                <filter token="log.location" value="D:/logs"/>
                <filter token="log.file.postfix" value="_2"/>
              </filterset>
            </copy>
          </tasks>
        </configuration>
        <goals>
          <goal>run</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

Note that Ant uses @ by default as delimiters for token (couldn't get it to use maven style delimiters) so the log4j.properties became:

log4j.appender.Application.File=@log.location@/application@log.file.postfix@.log
log4j.appender.tx_info.File=@log.location@/tx_info@log.file.postfix@.log

---

But, since these values seem to be node specific, did you consider using system properties instead (that you could place in the startup scripts)? This is something I've already done (with a log4j.xml), it works well and it would highly simplify things.