Improving Data Transfer Rate on Amazon EC2

Question

I\'ve got a 1-Gig EBS Volume mounted to an EC2 instance, I am copying 600MB of binary data from a Local Hard Drive (via RDS Connection) and the copying process windows is showi

Answer 1

Sometimes, if you want something done it's better to do it yourself :-)

I did not find anything satisfying over the net so I spent the evening doing a rather Complete Bench Test of my own. I have tested a few alternatives /scenarios and here are the results :

FTP server installed on EC2 was Filezilla Server (correct configuration is touchy)
FTP Client used for this Bench test was WinSPC (Filezilla Client didn't work. see this other post
Legend : [HC] stands for Home Connection (100MBps)

Upload Bandwidth
RDS Upload : 15 Kb/S => Worse Ever
FTP Upload [FTP Server Installed on EC2]: 100 Kb/S
Upload to S3 from AWS Management Console from HC: 60 Kb/S
Upload to S3 using AWS Console interface from EC2 : 145 Kb/S
Upload to S3 using S3 Browser from HC: 120 Kb/S
Upload to S3 using S3 Browser from EC2 : 2000 Kb/S

Download Bandwidth
RDS Download and Upload: 15 Kb/S => Worse Ever
FTP Download [FTP Server Installed on EC2]: 360 Kb/S
Download from S3 AWS Console interface from EC2 : 350 Kb/S
Download from S3 using S3 Browser: from HC: 380 to 620 Kb/S
Download from S3 using S3 Browser: from EC2 : 3000 Kb/S

Conclusions :

So, as of now, Amazon S3 combined with S3 Browser give the best results. (S3 Browser is just a layer over S3 I don't get it why the upload rate is better)

However, one should keep in mind that an FTP Server on an EC2 instance has the great advantage of Mapping directly a local EC2 Directory into EC2. Unlike S3, there is only one transfer involved. Indeed, S3 requires 2 Transfers : Form Local Resource to S3/and from S3 to EC2 and the other way round, while FTP Access grants immediacy by shortening transfer cycles. Besides, it spares the cost of S3 Buckets.

It is also interesting to mention that EC2 Instance's bandwidth is really strong. It is hence far more interesting -needless to say, to use protocols that really take advantage of it such as S3 or FTP, rather than RDS.

I hope this will be useful to other people facing the same issue and spare them precious time.

Answer 2

Use Aspera or Tsunami UDP and move 600 MB to your Amazon EC2 infra (jump box). Then internally copy from jumpbox to windows ec2. My benchmark shows tsunami UDP is quite faster compared to traditional modes.