Job Stealing Configuration not working in Apache Ignite

问题

I have the following configuration file

    <bean abstract="true" id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
    <property name="peerClassLoadingEnabled" value="true"/>

    <property name="includeEventTypes">
        <list>
            <!--Task execution events-->
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
            <util:constant static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
        </list>
    </property>

    <property name="metricsUpdateFrequency" value="10000"/>
    <property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="ipFinder">
                <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">
                    <property name="addresses">
                        <list>
                            <!-- In distributed environment, replace with actual host IP address. -->
                            <value>127.0.0.1:47500..47509</value>
                            <value>127.0.0.1:48500..48509</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>

      <!-- Enabling the required Failover SPI. -->
    <property name="failoverSpi">
        <bean class="org.apache.ignite.spi.failover.jobstealing.JobStealingFailoverSpi"/>
    </property>

    <property name="collisionSpi">
        <bean class="org.apache.ignite.spi.collision.jobstealing.JobStealingCollisionSpi">
            <property name="activeJobsThreshold" value="50"/>
            <property name="waitJobsThreshold" value="0"/>
            <property name="messageExpireTime" value="1000"/>
            <property name="maximumStealingAttempts" value="10"/>
            <property name="stealingEnabled" value="true"/>
        </bean>
    </property>
</bean>

The closure gets executed over the server nodes in the grid as expected.

When we add a new node by executing the below command to the grid during the execution of closure

The existing nodes acknowledge the addition of the new node in the grid but the closure is not distributed to the newly added node.

Below is my closure implementation

@Override
public AccruedSimpleInterest apply(SimpleInterestParameter simpleInterestParameter) {

    BigDecimal si = simpleInterestParameter.getPrincipal()
            .multiply(new BigDecimal(simpleInterestParameter.getYears()))
            .multiply(new BigDecimal(simpleInterestParameter.getRate())).divide(SimpleInterestClosure.HUNDRED);

    System.out.println("Calculated SI for id=" + simpleInterestParameter.getId() + " SI=" + si.toPlainString());
    return new AccruedSimpleInterest(si, simpleInterestParameter);
}

Below is the main class

public static void main(String... args) throws IgniteException, IOException {
    Factory<SimpleInterestClosure> siClosureFactory = FactoryBuilder.factoryOf(new SimpleInterestClosure());

    ClassPathResource ress = new ClassPathResource("example-ignite-poc.xml");
    File file = new File(ress.getPath());

    try (Ignite ignite = Ignition.start(file.getPath())) {
        System.out.println("Started Ignite Cluster");
        IgniteFuture<Collection<AccruedSimpleInterest>> igniteFuture = ignite.compute()
                .applyAsync(siClosureFactory.create(), createParamCollection());
        Collection<AccruedSimpleInterest> res = igniteFuture.get();
        System.out.println(res.size());
    }nter code here

回答1:

As far as my understanding goes, Job Stealing SPI requires you to implement some additional APIs in order to work.

Please see this discussion on user list:

Some remarks about job stealing SPI:

1)You have some nodes that can proceed the tasks of some compute job.

2)Tasks will be executed in public thread pool by default: https://apacheignite.readme.io/docs/thread-pools#section-public-pool

3)If some node thread pool is busy then some task of compute job can be executed on other node.

In next cases it will not work:

1)In case if you choose specific node for your compute task

2)In case if you do affinity call (the same as above but node will be choose by affinity mapping)

来源：https://stackoverflow.com/questions/58948793/job-stealing-configuration-not-working-in-apache-ignite

标签

ignite