问题
I'm using Java11 and httpclient 4.5.11 to make request to APIs, and so far everything works. But more than a week ago payments to https://api.moip.com.br started giving intermitent errors (Connection reset).
I know that they require TLSv1.2 and Java11 supports it, furthermore the errors are intermitent (somethimes it works, sometimes it doesn't). I contacted them and they said that they made a migration of their codebase, but that their configuration to process requests is the same, which I doubt, but in any case, considering that they process payments of several organizations, I assume that there is something that I can do to avoid these errors. As a side note, I can make requests without problems to PayPal, Stripe, Google Recaptcha, among other services.
How I create a connection:
try (CloseableHttpClient httpClient = createHttpClientBuilder().build()) {
HttpUriRequest httpMethod = createHttpUriRequest(request);
HttpResponse httpResponse = httpClient.execute(httpMethod);
SimpleHttpClientResponse response = createResponse(httpResponse);
return response;
} catch (Exception e) {
throw new HTTPException(request, e);
}
And the following are how I create the connection manager, the client builder and the request itself:
private static HttpClientBuilder createHttpClientBuilder() throws KeyManagementException, NoSuchAlgorithmException {
CookieStore httpCookieStore = new BasicCookieStore();
RequestConfig defaultRequestConfig = RequestConfig.custom().setCookieSpec(CookieSpecs.IGNORE_COOKIES).build();
HttpClientBuilder builder = HttpClients.custom()
.setConnectionManager(connectionManager)
.setConnectionManagerShared(true)
.setDefaultCookieStore(httpCookieStore)
.setDefaultRequestConfig(defaultRequestConfig);
return builder;
}
private static PoolingHttpClientConnectionManager createConnectionManager() {
try {
SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(
SSLContext.getDefault(),
new String[] {"TLSv1.2"},
null,
SSLConnectionSocketFactory.getDefaultHostnameVerifier());
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
.register("http", PlainConnectionSocketFactory.INSTANCE)
.register("https", socketFactory)
.build();
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(registry);
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(20);
return cm;
} catch (NoSuchAlgorithmException | RuntimeException e) {
LogUtils.error(e);
return null;
}
}
private static HttpUriRequest createHttpUriRequest(SimpleHttpClientRequest request) throws ParseException, IOException, URISyntaxException {
HttpUriRequest httpUriRequest;
if (request.getMethod().equals(HTTPMethod.POST)) {
HttpPost httpPost = new HttpPost(request.getUrl());
HttpEntity entity = createEntity(request);
httpPost.setEntity(entity);
httpUriRequest = httpPost;
} else if (request.getMethod().equals(HTTPMethod.HEAD)) {
httpUriRequest = new HttpHead(getUrlWithParameters(request));
} else if (request.getMethod().equals(HTTPMethod.DELETE)) {
httpUriRequest = new HttpDelete(getUrlWithParameters(request));
} else {
httpUriRequest = new HttpGet(getUrlWithParameters(request));
}
Map<String, String> headers = request.getHeaders();
if (headers != null) {
for (String key : headers.keySet()) {
String value = headers.get(key);
httpUriRequest.setHeader(key, value);
}
}
return httpUriRequest;
}
The SimpleHttpClientRequest is just a class of mine that defines the uri, headers, and other data to be used when making the request, but the request itself is done by apache HttpClient.
Like, I said, the errors are intermitent, if I try to make the same request several times it works.
I defined -Djavax.net.debug=all to know what could be the case, and the logs are like the following:
Obs: I've removed lines like ERROR [stderr] (default task-2) 0000: D2 A3 F4 C9 87 BF 23 89 65 F7 50 B6 90 8C 9D 8B ......#.e.P..... because I don't think they add anything usefult in this case
When I receive the error:
10:03:16,752 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23
10:03:16.750 GMT-03:00|SSLSocketInputRecord.java:249|READ: TLSv1.1 application_data, length = 2160
10:03:16,756 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:16.755 GMT-03:00|SSLCipher.java:1329|Padded plaintext after DECRYPTION (
10:03:16,775 ERROR [stderr] (default task-2) )
10:03:16,793 ERROR [stderr] (default task-2) javax.net.ssl|WARNING|DC|default task-2|2020-06-23 10:03:16.792 GMT-03:00|SSLSocketImpl.java:1280|handling exception (
10:03:16,793 ERROR [stderr] (default task-2) "throwable" : {
10:03:16,793 ERROR [stderr] (default task-2) java.net.SocketTimeoutException: Read timed out
10:03:16,793 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.socketRead0(Native Method)
10:03:16,794 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
10:03:16,794 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
10:03:16,802 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
...
10:03:16,812 ERROR [stderr] (default task-2) at java.base/java.lang.Thread.run(Thread.java:834)}
10:03:16,812 ERROR [stderr] (default task-2)
10:03:16,813 ERROR [stderr] (default task-2) )
10:03:16,820 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:16.813 GMT-03:00|SSLSocketOutputRecord.java:309|WRITE: TLS12 application_data, length = 329
10:03:16,821 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:16.821 GMT-03:00|SSLCipher.java:1743|Plaintext before ENCRYPTION (
10:03:16,827 ERROR [stderr] (default task-2) )
10:03:16,828 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:16.827 GMT-03:00|SSLSocketOutputRecord.java:323|Raw write (
10:03:16,829 ERROR [stderr] (default task-2) )
10:03:16,990 ERROR [stderr] (default task-2) javax.net.ssl|WARNING|DC|default task-2|2020-06-23 10:03:16.989 GMT-03:00|SSLSocketImpl.java:1280|handling exception (
10:03:16,990 ERROR [stderr] (default task-2) "throwable" : {
10:03:16,991 ERROR [stderr] (default task-2) java.net.SocketException: Connection reset
10:03:16,991 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
10:03:16,991 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
...
10:03:17,022 ERROR [stderr] (default task-2) at java.base/java.lang.Thread.run(Thread.java:834)}
10:03:17,022 ERROR [stderr] (default task-2)
10:03:17,022 ERROR [stderr] (default task-2) )
10:03:17,025 ERROR [stderr] (default task-2) javax.net.ssl|ERROR|DC|default task-2|2020-06-23 10:03:17.024 GMT-03:00|TransportContext.java:318|Fatal (UNEXPECTED_MESSAGE): Connection reset (
10:03:17,025 ERROR [stderr] (default task-2) "throwable" : {
10:03:17,025 ERROR [stderr] (default task-2) java.net.SocketException: Connection reset
10:03:17,026 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
10:03:17,026 ERROR [stderr] (default task-2) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
10:03:17,026 ERROR [stderr] (default task-2) at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
...
10:03:17,052 ERROR [stderr] (default task-2) at java.base/java.lang.Thread.run(Thread.java:834)}
10:03:17,052 ERROR [stderr] (default task-2)
10:03:17,052 ERROR [stderr] (default task-2) )
10:03:17,053 ERROR [stderr] (default task-2) javax.net.ssl|ALL|DC|default task-2|2020-06-23 10:03:17.053 GMT-03:00|SSLSessionImpl.java:784|Invalidated session: Session(1592917276199|TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256)
10:03:17,055 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.054 GMT-03:00|SSLSocketOutputRecord.java:71|WRITE: TLS12 alert(unexpected_message), length = 10
10:03:17,056 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.055 GMT-03:00|SSLCipher.java:1743|Plaintext before ENCRYPTION (
10:03:17,056 ERROR [stderr] (default task-2) )
10:03:17,059 ERROR [stderr] (default task-2) javax.net.ssl|WARNING|DC|default task-2|2020-06-23 10:03:17.059 GMT-03:00|TransportContext.java:360|Fatal: failed to send fatal alert UNEXPECTED_MESSAGE (
10:03:17,060 ERROR [stderr] (default task-2) "throwable" : {
10:03:17,060 ERROR [stderr] (default task-2) java.net.SocketException: Broken pipe (Write failed)
10:03:17,060 ERROR [stderr] (default task-2) at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
10:03:17,061 ERROR [stderr] (default task-2) at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
...
10:03:17,076 ERROR [stderr] (default task-2) at java.base/java.lang.Thread.run(Thread.java:834)}
10:03:17,076 ERROR [stderr] (default task-2)
10:03:17,076 ERROR [stderr] (default task-2) )
10:03:17,077 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.077 GMT-03:00|SSLSocketImpl.java:1353|close the underlying socket
10:03:17,077 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.077 GMT-03:00|SSLSocketImpl.java:1372|close the SSL connection (initiative)
10:03:17,078 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.078 GMT-03:00|SSLSocketImpl.java:663|close outbound of SSLSocket
10:03:17,078 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:03:17.078 GMT-03:00|SSLSocketImpl.java:629|close inbound of SSLSocket
10:03:17,079 ERROR [stderr] (default task-2) javax.net.ssl|WARNING|DC|default task-2|2020-06-23 10:03:17.079 GMT-03:00|TransportContext.java:284|Closed transport, general or untracked problem
10:03:17,105 WARN [my.company.util.LogUtils] (default task-2)
at my.deployment//my.company.util.ExceptionUtils.wrap(ExceptionUtils.java:26)
at my.deployment//my.company.util.PagamentoUtil.consultarRespostaHttpPagamento(PagamentoUtil.java:618)
at my.deployment//my.company.controller.admin.verification.VerifyPaymentApiController.doGet(VerifyPaymentApiController.java:61)
... 58 more
Caused by: my.company.exceptions.HTTPException: javax.net.ssl.SSLException: Connection reset - 'GET': 'https://api.moip.com.br/v2/orders/ORD-KC1CSM2SVPMX'
at my.deployment//my.company.util.HttpClientUtils.send(HttpClientUtils.java:63)
at my.deployment//my.company.util.URIUtils.getResponseObjectFromRequest(URIUtils.java:315)
at my.deployment//my.company.util.PagamentoUtil.accessMoipV2(PagamentoUtil.java:2060)
at my.deployment//my.company.util.PagamentoUtil.lambda$consultarRespostaHttpPagamento$0(PagamentoUtil.java:597)
at my.deployment//my.company.util.PagamentoUtil.getResponseResultFromApi(PagamentoUtil.java:806)
at my.deployment//my.company.util.PagamentoUtil.consultarRespostaHttpPagamento(PagamentoUtil.java:597)
... 59 more
Caused by: javax.net.ssl.SSLException: Connection reset
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:269)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1306)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:832)
at my.deployment//org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at my.deployment//org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at my.deployment//org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at my.deployment//org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at my.deployment//org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at my.deployment//org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at my.deployment//org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at my.deployment//org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at my.deployment//org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at my.deployment//org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at my.deployment//org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at my.deployment//org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at my.deployment//org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at my.deployment//org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at my.deployment//org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at my.deployment//org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at my.deployment//org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at my.deployment//my.company.util.HttpClientUtils.send(HttpClientUtils.java:59)
... 64 more
Suppressed: java.net.SocketException: Broken pipe (Write failed)
at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:357)
... 86 more
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:68)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1096)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:816)
... 82 more
When it errors, it doesn't reach the part in the successful request that logs:
10:01:13,234 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:01:13.233 GMT-03:00|TrustStoreManager.java:161|Inaccessible trust store: /usr/lib/jvm/java-11-openjdk-11.0.7.10-4.el7_8.x86_64/lib/security/jssecacerts
10:01:13,234 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:01:13.234 GMT-03:00|TrustStoreManager.java:112|trustStore is: /usr/lib/jvm/java-11-openjdk-11.0.7.10-4.el7_8.x86_64/lib/security/cacerts
10:01:13,234 ERROR [stderr] (default task-2) trustStore type is: pkcs12
10:01:13,234 ERROR [stderr] (default task-2) trustStore provider is:
10:01:13,235 ERROR [stderr] (default task-2) the last modified time is: Thu Jun 11 13:31:42 GMT-03:00 2020
10:01:13,553 ERROR [stderr] (default task-2) javax.net.ssl|DEBUG|DC|default task-2|2020-06-23 10:01:13.551 GMT-03:00|X509TrustManagerImpl.java:79|adding as trusted certificates (
10:01:13,554 ERROR [stderr] (default task-2) "certificate" : {
10:01:13,554 ERROR [stderr] (default task-2) "version" : "v3",
I saw the SocketTimeoutException (even tough it happens only 200ms after the request begins) and tried to increase the socket timeout either in the connection manager as well as in the client builder, and also in the request, but it didn't work. I also tried to remove the keepalive and make it so that the connections aren't reused, but it also didn't work. I also looked at other questions here in SO with a similar problem and none worked. The following are the changes I tried:
In the connection manager:
SocketConfig socketConfig = SocketConfig.custom()
.setSoKeepAlive(false)
.setSoTimeout(600000)
.setSoReuseAddress(false)
.build();
cm.setDefaultSocketConfig(socketConfig);
In the client:
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(15000)
.setConnectTimeout(15000)
.setCookieSpec(CookieSpecs.IGNORE_COOKIES).build();
builder = builder.setDefaultRequestConfig(requestConfig);
builder = builder.setConnectionReuseStrategy((response, context) -> {
LogUtils.info("**** connectionReuse strategy returning false");
return false;
});
builder = builder.setKeepAliveStrategy((response, context) -> {
LogUtils.info("**** keepAlive strategy returning -1");
return -1l;
});
In the request:
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(15000)
.setConnectTimeout(15000)
.setCookieSpec(CookieSpecs.IGNORE_COOKIES).build();
httpUriRequest.setConfig(requestConfig);
I tried the above together, as well as individually, and none worked.
Something I realized is that I never received the exception right after started the java application when testing locally (could be a coincidence, tough), and when a request works, the following requests work for 1 minute or so, but even when I defined to not reuse connections it continued the same way, with intermittent errors, and working for a time if one request is successful (although maybe that configuration didn't disabled the reuse of connections, but I think it disabled because I saw the logs with connectionReuse strategy returning false), so I ended up out of options.
For now I included them in a while loop that executes at most 4 times (when it errors) to make the request to that specific endpoint work seamlessly to the users, but this is a very hack approach and I would like to fix this problem, if possible.
I hope someone can help me with this.
来源：https://stackoverflow.com/questions/62538867/java-11-connection-reset-intermitent