InfinispanDirectoryProvider with Wildfly 10.1

Question

We would like to run our EAR, which uses hibernate search, in a HA cluster with wildfly 10.1 and jgroups using infinispan as a cache for hibernate and hibernate search. The basic cluster configuration, based on standalone-full-ha.xml (additional details below) is working when we use:

<property name="hibernate.search.default.directory_provider" value="ram"/>

for Hibernate Search's directory provider. But when we change this to:

<property name="hibernate.search.default.directory_provider" value="infinispan"/>

We get errors:

01:48:21,857 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 80) MSC000001: Failed to start service jboss.persistenceunit."mc.ear/web.war#mc": org.jboss.msc.service.StartException in service jboss.persistenceunit."mc.ear/web.war#mc": javax.persistence.PersistenceException: [PersistenceUnit: mc] Unable to build Hibernate SessionFactory
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:179)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:121)
    at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:667)
    at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:193)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: javax.persistence.PersistenceException: [PersistenceUnit: mc] Unable to build Hibernate SessionFactory
     /* SNIP */
Caused by: org.hibernate.search.exception.SearchException: Unable to find directory provider implementation class: org.infinispan.hibernate.search.spi.InfinispanDirectoryProvider

I know that infinispan directory provider is no longer distributed with WildFly (currently using 10.1.0-Final), but I cannot figure out how to add it as a module and reference it in my jboss-deployment-structure.xml. It seems it should not be that hard.

I tried the advise here to disable version detection, but I don't think that is the problem: https://developer.jboss.org/thread/267716

In fact, I tried this a while back with WildFly 10.0 but also ran into the same issues: https://developer.jboss.org/thread/271789

I guess I don't know how to include the infinispan directory provider as custom module - do I need to completely replace infinispan? Doesn't this have an effect on the other caches in WildFly? I would like to minimize the changes from the standard, but am really having trouble. Has anyone had success integrating InfinispanDirectoryProvider with Wildfly 10.1 and could provide a brief description here? Additional details below:

my persistence.xml

<persistence
    xmlns="http://xmlns.jcp.org/xml/ns/persistence"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/persistence
        http://xmlns.jcp.org/xml/ns/persistence/persistence_2_1.xsd"
    version="2.1">
  <persistence-unit name="mc">
    <jta-data-source>java:/mcDS</jta-data-source>
    <properties>
      <property name="hibernate.show_sql"                       value="false"/>
      <property name="hibernate.id.new_generator_mappings"      value="false"/>
      <property name="hibernate.cache.use_second_level_cache"   value="true"/>
      <property name="hibernate.cache.use_query_cache"          value="true"/>
      <property name="jboss.entity.manager.jndi.name"           value="java:jboss/EntityManagers/mc"/> 
      <property name="jboss.entity.manager.factory.jndi.name"   value="java:jboss/EntityManagerFactories/mc"/>
      <property name="wildfly.jpa.hibernate.search.module" value="none" />
    </properties>
  </persistence-unit>
</persistence>

I have been configuring the other relevant settings directly in the standalone.xml:

<system-properties>
    <property name="hibernate.search.default.worker.backend" value="jgroups"/>
    <property name="hibernate.search.default.directory_provider" value="infinispan"/>
    <property name="hibernate.search.default.exclusive_index_use" value="false"/>
    <property name="hibernate.search.infinispan.chunk_size" value="300000000"/>
    <property name="hibernate.search.reader.strategy" value="shared"/>
    <property name="hibernate.search.lucene_version" value="LUCENE_CURRENT"/>
    <property name="hibernate.search.worker.execution" value="async"/>
    <property name="hibernate.search.infinispan.cachemanager_jndiname" value="java:jboss/infinispan/container/hibernateSearch"/>
</system-properties>

Answer 1

The short answer is:

• get the right zip file
• adjust your configuration accordingly

Detailed explanation:

To use the Infinispan Lucene Directory in WildFly you need to download the Infinispan distribution of its custom modules for WildFly; you can find them on the Infinispan downloads page, it's the zip named "WildFly/EAP Modules".

Uncompress this zip file in your WildFly /modules directory. It might help to look at what this adds: among others you will find this contains a module named "org.infinispan.hibernate-search.directory-provider" and having slot "for-hibernatesearch-5.5".

It will also contain a new copy of Infinispan, but since it uses a different module "slot" than the one included in WildFly, you're not overwriting it. Existing applications and WidlFly's own usage of Infinispan will still be ablet to use the existing module, as classloader isolation actually works.

Within WildFly the Hibernate Search module "org.hibernate.search.engine" has an optional dependency on this module you just added, so if you use the Hibernate Search version included in WildFly you're done!

You configuration isn't using the Wildfly module though; remove the line "wildfly.jpa.hibernate.search.module" and remove the Hibernate Search jars from your deployment if you have them there.

Keep in mind that you can't configure the Caches of this Infinispan instance using your WildFly's "standalone.xml": you're adding an additional, different module of Infinispan. This could for example be a more recent version of Infinispan than the one included with Wildfly, if you want to. So make sure that those JNDI names do not refer to the Infinispan instance which is bundled with WildFly. The easiest solution might be to not use the JNDI configuration strategy, but to simply include an Infinispan configuration file with your application deployment, and have Hibernate Search refer to that resource to start a new data grid.

Since noone likes to manually download and unzip archives, you can download the modules zip file via Maven from coordinates:

<groupId>org.infinispan</groupId>
<artifactId>infinispan-as-embedded-modules</artifactId>
<version>${infinispan.version}</version>
<type>zip</type>

For example, here you can find how the Infinispan testsuite runs integration tests automatically of this module itself: https://github.com/infinispan/infinispan/blob/8.2.5.Final/integrationtests/as-lucene-directory/pom.xml#L157-L204