How to improve the Performance of FtpWebRequest?

Question

I have an application written in .NET 3.5 that uses FTP to upload/download files from a server. The app works fine but there are performance issues:

1. It take

Answer 1

I'd recommend switching to rsync.
Pros :
Optimised for reducing transfer time.
Supports SSH for secure transfer
Uses TCP so makes your IT dept/firewall guys happier

Cons:
No native .NET support
Geared towards linux server installations - though there are decent windows ports like DeltaCopy

Overall though it's a much better choice than FTP

Answer 2

i was working a few days with that... and speed was really low, nothing to compare with FileZilla... finally i solved with multithreads. 10 threads making connections for download gives me a good rate, maybe even could be improved more.. with a standar ftprequest configuration

PeticionFTP.ConnectionGroupName = "MyGroupName"
PeticionFTP.ServicePoint.ConnectionLimit = 4
PeticionFTP.ServicePoint.CloseConnectionGroup("MyGroupName")

PeticionFTP.KeepAlive = False 
PeticionFTP.UsePassive = False

PeticionFTP.UseBinary = True

PeticionFTP.Credentials = New NetworkCredential(lHost.User, lHost.Password)

Answer 3

It doesn't matter if the individual connections take long to connect as long as you can launch many in parallel. If you have many items to transfer (say hundreds) then it makes sense to launch tens and even hundreds of WebRequests in parallel, using the asynchronous methods like BeginGetRequestStream and BeginGetResponse. I worked on projects that faced similar problems (long connect/authenticate times) but by issuing many calls in parallel the overall throughput was actually very good.

Also it makes a huge difference if you use the async methods or the synchronous one, as soon as you have many (tens, hundreds) of requests. This applies not only to your WebRequests methods, but also to your Stream read/write methods you'll use after obtaining the upload/download stream. The Improving .Net Performance and Scalability book is a bit outdated, but much of its advice still stands, and is free to read online.

One thing to consider is that the ServicePointManager class sits there lurking in the Framwework with one sole purpose: to ruin your performance. Make sure you obtain the ServicePoint of your URL and change the ConnectionLimit to a reasonable value (at least as high as how many concurrent requests you intend).

Answer 4

I strongly suggest Starksoft FTP/FTPS Component for .NET and Mono. It has a connection object that you can cache and reuse.

Answer 5

I did some benchmarks on FtpWebRequest, similar to @Sid 's response above. Setting KeepAlive to true does improve, but not the asynchronous calls in my test. The test consists of

1) FtpWebRequest for check of file existence 2) FtpWebRequest for upload 3) FtpWebRequest for rename file on server

Test FTP client 30 files of size 5 Kbytes took ... 14.897 seconds Test upload (alive true, connection name) 30 files of size 5 Kbytes took ... 34.229 seconds Test async(alive true, connection name) 30 files of size 5 Kbytes took ... 40.659 seconds Test send thread (alive true, connection name) 30 files of size 5 Kbytes took ... 38.926 seconds, 30 files

what did improve was an implementation of an FTP client made using the Socket class

the benchmark is here

https://github.com/pedro-vicente/ftp_t

Answer 6

I have had good results with EDT's ftp library:

http://www.enterprisedt.com/products/edtftpnet/overview.html