spring xd losing messages when processing huge volume
I am using spring xd My stream looks like below and running tests on 3 node container with 1 admin node with rabbit as transport aws-s3|processor1|http-client|processor2>queue:readyQueue I have created below tap. tap1 aws-s3>s3Queue tap2 processor1>processorQueue1 tap3 http-client>httpQueue I run below scenarios in my tests: Scenario1 : 5 files of 200k =1 Million records concurrency of http-client=70 and processor2=30 I see 900k message s3Queue I see 889k message processorQueue1 I see 886k message httpQueue I see 883k message processorQueue2 Messages are lost everywhere and its random