问题
Lets consider following code:
Client code:
public class MyClient {
private final MyClientSideService myClientSideService;
public MyClient(MyClientSideService myClientSideService) {
this.myClientSideService = myClientSideService;
}
public String requestRow(Integer req) {
return myClientSideService.requestSingleRow(req);
}
}
Client side service:
public class MyClientSideService {
private final MyServerSideService myServerSideService;
public MyClientSideService(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
return myServerSideService.requestRowBatch(Arrays.asList(req)).get(0);
}
}
Server side service:
@Slf4j
public class MyServerSideService {
//single threaded bottleneck service
public synchronized List<String> requestRowBatch(List<Integer> batchReq) {
log.info("Req for {} started");
try {
Thread.sleep(100);
return batchReq.stream().map(String::valueOf).collect(Collectors.toList());
} catch (InterruptedException e) {
return null;
} finally {
log.info("Req for {} finished");
}
}
}
And main:
@Slf4j
public class MainClass {
public static void main(String[] args) {
MyClient myClient = new MyClient(new MyClientSideService(new MyServerSideService()));
for (int i = 0; i < 20; i++) {
new Thread(() -> {
for (int m = 0; m < 100; m++) {
int k = m;
log.info("Response is {}", myClient.requestRow(k));
}
}).start();
}
}
}
According the logs it takes approximately 4 min 22 sec but it too much. Ithink it might be improved dramatically. I would like to implement implicit batching. So MyClientSideService should collect requests and when it becomes 50(it is preconfigured batch size) or some preconfigured timeout expired then to request MyServerSideService and back route result to the clients. Protocol should be synchronous so clients must be blocked until result getting.
I tried to write code using CountDownLatches and CyclicBarriers but my attempts were far from success.
How can I achieve my goal?
P.S.
If to replace requestRowBatch return type List<String> from to Map<Integer, String> to delegate request and response mapping to server following works with limititations. It works only if I send <=25 requests
@Slf4j
public class MyClientSideService {
private final Integer batchSize = 25;
private final Integer maxTimeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final Queue<Integer> queue = new ArrayBlockingQueue(batchSize);
private final Map<Integer, String> responseMap = new ConcurrentHashMap();
private final AtomicBoolean started = new AtomicBoolean();
private CountDownLatch startBatchRequestLatch = new CountDownLatch(batchSize);
private CountDownLatch awaitBatchResponseLatch = new CountDownLatch(1);
public MyClientSideService(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
queue.offer(req);
if (!started.compareAndExchange(false, true)) {
log.info("Start batch collecting");
startBatchCollecting();
}
startBatchRequestLatch.countDown();
try {
log.info("Awaiting batch response latch for {}...", req);
awaitBatchResponseLatch.await();
log.info("Finished awaiting batch response latch for {}...", req);
return responseMap.get(req);
} catch (InterruptedException e) {
e.printStackTrace();
return "EXCEPTION";
}
}
private void startBatchCollecting() {
new Thread(() -> {
try {
log.info("Await startBatchRequestLatch");
startBatchRequestLatch.await(maxTimeoutMillis, TimeUnit.MILLISECONDS);
log.info("await of startBatchRequestLatch finished");
} catch (InterruptedException e) {
e.printStackTrace();
}
responseMap.putAll(requestBatch(queue));
log.info("Releasing batch response latch");
awaitBatchResponseLatch.countDown();
}).start();
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
Update
According Malt answer I was able to develop following:
@Slf4j
public class MyClientSideServiceCompletableFuture {
private final Integer batchSize = 25;
private final Integer maxTimeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final Queue<Pair<Integer, CompletableFuture>> queue = new ArrayBlockingQueue(batchSize);
private final AtomicInteger counter = new AtomicInteger(0);
private final Lock lock = new ReentrantLock();
public MyClientSideServiceCompletableFuture(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
lock.lock();
try {
queue.offer(Pair.of(req, future));
int counter = this.counter.incrementAndGet();
if (counter != 0 && counter % batchSize == 0) {
log.info("request");
List<Integer> requests = queue.stream().map(p -> p.getKey()).collect(Collectors.toList());
Map<Integer, String> serverResponseMap = requestBatch(requests);
queue.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
queue.clear();
}
} finally {
lock.unlock();
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
But it doesn't work if size is not multiple of batch size
回答1:
If to replace requestRowBatch return type from List<String> with Map<Integer, String> to delegate request and response mapping to server I was able to crete following solution:
@Slf4j
public class MyClientSideServiceCompletableFuture {
private final Integer batchSize = 25;
private final Integer timeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final BlockingQueue<Pair<Integer, CompletableFuture>> queue = new LinkedBlockingQueue<>();
private final Lock lock = new ReentrantLock();
private final Condition requestAddedCondition = lock.newCondition();
public MyClientSideServiceCompletableFuture(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
startQueueDrainer();
}
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
while (!queue.offer(Pair.of(req, future))) {
log.error("Can't add {} to the queue. Retrying...", req);
}
lock.lock();
try {
requestAddedCondition.signal();
} finally {
lock.unlock();
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}
private void startQueueDrainer() {
new Thread(() -> {
log.info("request");
while (true) {
ArrayList<Pair<Integer, CompletableFuture>> requests = new ArrayList<>();
if (queue.drainTo(requests, batchSize) > 0) {
log.info("drained {} items", requests.size());
Map<Integer, String> serverResponseMap = requestBatch(requests.stream().map(Pair::getKey).collect(Collectors.toList()));
requests.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
} else {
lock.lock();
try {
while (queue.size() == 0) {
try {
log.info("Waiting on condition");
requestAddedCondition.await(timeoutMillis, TimeUnit.MILLISECONDS);
log.info("Waking up on condition");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} finally {
lock.unlock();
}
}
}
}).start();
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
It looks like a working solution. But I am not sure if it is optimal.
回答2:
Your MyClientSideServiceCompletableFuture solution, will send the requests to the server every time you add something to the queue and doesnt wait for requests to be batch sized. You are using BlockingQueue and adding the uneccessary blocking condition and locks. BlockingQueue has blocking-timeout capabilites so no addition condition is neccessary.
You can simplify your solution like this:
It sends requests to server only when the batch is full or the timeout passed and batch is not empty.
private void startQueueDrainer() {
new Thread(() -> {
log.info("request");
ArrayList<Pair<Integer, CompletableFuture>> batch = new ArrayList<>(batchSize);
while (true) {
try {
batch.clear(); //clear batch
long timeTowWait = timeoutMillis;
long startTime = System.currentTimeMillis();
while (timeTowWait > 0 && batch.size() < batchSize) {
Pair<Integer, CompletableFuture> request = queue.poll(timeTowWait , TimeUnit.MILLISECONDS);
if(request != null){
batch.add(request);
}
long timeSpent = (System.currentTimeMillis() - startTime);
timeTowWait = timeTowWait - timeSpent;
}
if (!batch.isEmpty()) {
// we wait at least timeoutMillis or batch is full
log.info("send {} requests to server", batch.size());
Map<Integer, String> serverResponseMap = requestBatch(batch.stream().map(Pair::getKey).collect(Collectors.toList()));
batch.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
} else {
log.info("We wait {} but the batch is still empty", System.currentTimeMillis() - startTime);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
}
Change the method requestSingleRow to not use lock
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
while (!queue.offer(Pair.of(req, future))) {
log.error("Can't add {} to the queue. Retrying...", req);
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}
回答3:
You could use CompletableFuture.
Have threads calling MyClientSideService put their request in a Queue (possibly BlockingQueue, and get a new CompletableFuture in return. The calling thread can call CompletableFuture.get() to block until a result is ready, or go on doing other things.
That CompletableFuture will be stored together with the request in MyClientSideService. When you reach 50 requests (and therefore 50 CompletableFuture instances), have the client service send the batch request.
When the request is complete, use the CompletableFuture.complete(value) method of each ComplatableFuture instance in the queue to notify the client thread that the response is ready. This will unblock the client if it has called blocking method like CompletableFuture.get(), or make it return instantly with value if called later.
来源：https://stackoverflow.com/questions/58666190/how-to-add-batching-implicit-for-client