Indy error 10038 “Socket operation on non-socket” after 61 seconds of inactivity

Question

I want to download some large files (GB) from an FTP server. The download of the first file always works. Then when trying to get the second file I get:

Answer 1

FTP uses multiple socket connections, one for commands/responses, and separate connections for each transfer.

The most likely cause of the socket error is an FTP-unaware proxy/router/firewall sitting in between TIdFTP and the FTP server is closing the command connection after a short period of inactivity. During the Unpack() (or manual pause), no commands/responses are being transmitted on the command connection, it is sitting idle, and thus is subject to being closed by a timeout on such a proxy/router/firewall.

During a transfer, the command connection is sitting idle, no FTP commands/responses are being transmitted on it (unless you abort the transfer), until the transfer is complete. An FTP-unaware proxy/router/firewall may close the command connection prematurely during this time.

To avoid that, TIdFTP has a NATKeepAlive property that can enable TCP keep-alives on the command connection while it is sitting idle. This usually prevents premature closes.

However, when there is no transfer in prgress, TIdFTP disables TCP keep-alives on the command connection if NATKeepAlive.UseKeepAlive is True. TIdFTP uses TCP keep-alives only during transfers, with the assumption that you are not going to perform long delays in between FTP commands. If you need to delay for awhile, either close the FTP connection, or send an FTP command at regular intervals (such as calling TIdFTP.Noop() in a timer/thread).

Alternatively, you can try manually enabling TCP keep-alives after connecting to the server, and set NATKeepAlive.UseKeepAlive to False so TIdFTP will not automatically disable the keep-alives after each transfer, eg:

function Connect2FTP(FTP: TIdFTP; RemoteFolder: string; Log: TRichLog): Boolean;      
begin
  Result := FTP.Connected;
  if not Result then
  begin         { We are not already connected }
    FTP.Host    := MyFTP;
    FTP.Username:= usr;
    FTP.Password:= psw;

    try
      FTP.Connect;
      try
        FTP.ChangeDir(RemoteFolder); 

        // send a TCP keep-alive every 5 seconds after being idle for 10 seconds
        FTP.NATKeepAlive.UseKeepAlive := False; // False by default, but just in case...
        FTP.Socket.Binding.SetKeepAliveValues(True, 10000, 5000);
      except
        FTP.Disconnect(False);
        raise;
      end;
    except
      Exit;
    end;

    Result := True;
  end;
end;

Note that while most platforms support enabling TCP keep-alives on a per-connection basis (keep-alives are part of the TCP spec), setting keep-alive intervals is platform-specific. At this time, Indy supports setting the intervals on:

• Windows 2000+ and WinCE 4.x+, for Win32 and .NET
• Linux, when Indy is used in Kylix
• Unix/Linux and NetBSD, when Indy is used in FreePascal.

Otherwise, OS default intervals get used, which may or may not be too large in value for this situation, depending on OS configuration.

At this time, the intervals are not customizable in Delphi FireMonkey, except for Windows.

If a particular platform supports setting custom TCP keep-alive intervals, but Indy does not implement them in SetKeepAliveValues(), you can use TIdFTP.Socket.Binding.SetSockOpt() instead to set the values manually as needed. Many platforms do support TCP_KEEPIDLE/TCP_KEEPINTVL or equivalent socket options.