Environment: I'm using Sun Java JDK 1.8.0_60 on 64-bit Windows 7, using Spring Integration 4.1.6 (which internally appears to use Apache Commons Net 3.3 for FTPS access).
I'm attempting to integrate with our application an automatic download from our client's FTPS server. I've done so successfully with SFTP servers using Spring Integration without any trouble for other clients without issues, but this is the first time a client has required us to use FTPS, and getting it to connect has been very puzzling. While in my real application I'm configuring Spring Integration using XML beans, to try to understand what's not working I'm using the following test code (though I'm anonymizing the actual host/username/password here):
final DefaultFtpsSessionFactory sessionFactory = new DefaultFtpsSessionFactory();
sessionFactory.setHost("XXXXXXXXX");
sessionFactory.setPort(990);
sessionFactory.setUsername("XXXXXXX");
sessionFactory.setPassword("XXXXXXX");
sessionFactory.setClientMode(2);
sessionFactory.setFileType(2);
sessionFactory.setUseClientMode(true);
sessionFactory.setImplicit(true);
sessionFactory.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager());
sessionFactory.setProt("P");
sessionFactory.setProtocol("TLSv1.2");
sessionFactory.setProtocols(new String[]{"TLSv1.2"});
sessionFactory.setSessionCreation(true);
sessionFactory.setCipherSuites(new String[]{"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"});
final FtpSession session = sessionFactory.getSession();
//try {
final FTPFile[] ftpFiles = session.list("/");
logger.debug("FtpFiles: {}", (Object[]) ftpFiles);
//} catch (Exception ignored ) {}
session.close();
I'm running this code with -Djavax.net.debug=all
to get all the TLS debugging information printed.
The main "control" connection to the FTPS server works fine, but when it tries to open the data connection for the list (or any other data connection I've tried), I get a javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
, caused by java.io.EOFException: SSL peer shut down incorrectly
. If I uncomment the swallowing-exceptions catch block around the session.list
command, then I can see (though the javax.net.debug output) that the server sent the following message after rejecting the data connection SSL handshake:
main, READ: TLSv1.2 Application Data, length = 129
Padded plaintext after DECRYPTION: len = 105
0000: 34 35 30 20 54 4C 53 20 73 65 73 73 69 6F 6E 20 450 TLS session
0010: 6F 66 20 64 61 74 61 20 63 6F 6E 6E 65 63 74 69 of data connecti
0020: 6F 6E 20 68 61 73 20 6E 6F 74 20 72 65 73 75 6D on has not resum
0030: 65 64 20 6F 72 20 74 68 65 20 73 65 73 73 69 6F ed or the sessio
0040: 6E 20 64 6F 65 73 20 6E 6F 74 20 6D 61 74 63 68 n does not match
0050: 20 74 68 65 20 63 6F 6E 74 72 6F 6C 20 63 6F 6E the control con
0060: 6E 65 63 74 69 6F 6E 0D 0A nection..
What appears to be happening (and this is my first time dealing with FTPS, though I've dealt with plain FTP before) is that the way the server ensures authentication and encryption over both the control and data connections is that after a "normal" TLS connection to establish the control connection and authentication happens there, each data connection requires the client to connect with the same TLS session. This makes sense to me as how it's supposed to be work, but the Apache Commons Net FTPS implementation doesn't seem to be doing that. It seems to be trying to establish a new TLS session, and so the server is rejecting the attempt.
Based on this question about resuming SSL sessions in JSSE, it appears that Java assumes or requires a different session for each host/post combination. My hypothesis is that since the FTPS data connection is on a different port than the control connection, it's not finding the existing session and is trying to establish a new one, so the connection fails.
I see three main possibilities:
- The server is not following the FTPS standard in requiring the same TLS session on the data port as on the control port. I can connect to the server fine (using the same host/user/password as I'm trying to use in my code) using FileZilla 3.13.1. The server identifies itself as "FileZilla Server 0.9.53 beta" upon login, so perhaps this is some sort of proprietary FileZilla way of doing things, and there's something odd I need to do to convince Java to use the same TLS session.
- The Apache Commons Net client doesn't actually follow the FTPS standard, and only allows some subset that doesn't allow for securing the data connections. This would seem odd, as it appears to be the standard way of connecting to FTPS from within Java.
- I'm completely missing something and misdiagnosing this.
I'd appreciate any direction you can provide as to how to connect to this kind of FTPS server. Thank you.
Indeed some FTP(S) servers do require that the TLS/SSL session is reused for the data connection. This is a security measure by which the server can verify that the data connection is used by the same client as the control connection.
Some references for common FTP servers:
- vsftpd: https://scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html
- FileZilla server: https://svn.filezilla-project.org/filezilla?view=revision&revision=6661
- ProFTPD: http://www.proftpd.org/docs/contrib/mod_tls.html#TLSOptions (
NoSessionReuseRequired
directive)
What may help you with the implementation is that Cyberduck FTP(S) client does support TLS/SSL session reuse and it uses Apache Commons Net library:
https://trac.cyberduck.io/ticket/5087 - Reuse Session key on data connection
See its
FTPClient.java
code (extends Commons NetFTPSClient
), particularly its override of_prepareDataSocket_
method:@Override protected void _prepareDataSocket_(final Socket socket) throws IOException { if(preferences.getBoolean("ftp.tls.session.requirereuse")) { if(socket instanceof SSLSocket) { // Control socket is SSL final SSLSession session = ((SSLSocket) _socket_).getSession(); if(session.isValid()) { final SSLSessionContext context = session.getSessionContext(); context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size")); try { final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache"); sessionHostPortCache.setAccessible(true); final Object cache = sessionHostPortCache.get(context); final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class); method.setAccessible(true); method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session); method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session); } catch(NoSuchFieldException e) { // Not running in expected JRE log.warn("No field sessionHostPortCache in SSLSessionContext", e); } catch(Exception e) { // Not running in expected JRE log.warn(e.getMessage()); } } else { log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket)); } } } }
It seems that the
_prepareDataSocket_
method was added to Commons NetFTPSClient
specifically to allow the TLS/SSL session reuse implementation:
https://issues.apache.org/jira/browse/NET-426A native support for the reuse is still pending:
https://issues.apache.org/jira/browse/NET-408You will obviously need to override the Spring Integration
DefaultFtpsSessionFactory.createClientInstance()
to return your customFTPSClient
implementation with the session reuse support.
The above solution does not work on its own anymore since JDK 8u161.
According to JDK 8u161 Update Release Notes (and the answer by @Laurent):
Added TLS session hash and extended master secret extension support
...
In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property
jdk.tls.useExtendedMasterSecret
tofalse
in the JDK
I.e., you can call this to fix the problem:
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
Though this should be a considered a workaround only. I do not know a proper solution.
You can use this SSLSessionReuseFTPSClient class :
import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.Socket;
import java.util.Locale;
import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSessionContext;
import javax.net.ssl.SSLSocket;
import org.apache.commons.net.ftp.FTPSClient;
public class SSLSessionReuseFTPSClient extends FTPSClient {
// adapted from:
// https://trac.cyberduck.io/browser/trunk/ftp/src/main/java/ch/cyberduck/core/ftp/FTPClient.java
@Override
protected void _prepareDataSocket_(final Socket socket) throws IOException {
if (socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket) _socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
method.setAccessible(true);
method.invoke(cache, String
.format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT), session);
method.invoke(cache, String
.format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort()))
.toLowerCase(Locale.ROOT), session);
} catch (NoSuchFieldException e) {
throw new IOException(e);
} catch (Exception e) {
throw new IOException(e);
}
} else {
throw new IOException("Invalid SSL Session");
}
}
}
}
And With openJDK 1.8.0_161 :
We must set :
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
according to http://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html
Added TLS session hash and extended master secret extension support
In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property jdk.tls.useExtendedMasterSecret to false in the JDK
To make Martin Prikryl's suggestion work for me I had to store the key not only under socket.getInetAddress().getHostName()
but also under socket.getInetAddress().getHostAddress()
.
(Solution stolen from here.)
来源:https://stackoverflow.com/questions/32398754/how-to-connect-to-ftps-server-with-data-connection-using-same-tls-session