Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow SFTP transfers #12134

Closed
cyberduck opened this issue Feb 3, 2006 · 65 comments
Closed

Slow SFTP transfers #12134

cyberduck opened this issue Feb 3, 2006 · 65 comments
Assignees
Labels
Milestone

Comments

@cyberduck
Copy link
Collaborator

11e0fc1 created the issue

If I use the command line sftp client on the local network, I can get transfer speeds of several MB/s.

If I use CyberDuck, the fastest it gets is 600KB/s, which was after I tried increasing the buffer size to 65536.

Is this a known problem? Cyberduck is still nice, but useless for local networks...


Attachments

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Duplicate for #12072.

@cyberduck
Copy link
Collaborator Author

3db2538 commented

This is not a dupe of #12072. That one says that CyberDuck doesn't support compression, resulting in slower transfers. While that may be true for WAN transfers, adding compression over LAN transfers will actually slow them down.

The issue is that CyberDuck is about 10x slower than the sftp client on a LAN, with compression disabled.

Isn't this a really easy one to verify?

@cyberduck
Copy link
Collaborator Author

Michael Schubart commented

I noticed the same thing. Cyberduck: A few 100 KByte/s. Command line scp: a few MByte/s.

@cyberduck
Copy link
Collaborator Author

mfichtner commented

The problem still seems to exist in the latest pre-release 2.5.5. Download speed through SFTP is still approx. 10% of the available bandwidth.

@cyberduck
Copy link
Collaborator Author

analogue commented

Same for me, using the 2.5.5, I get ~85KB/s using Cyberduck, and ~400KB/s using the command line sftp client.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

This is a bug in the SSH I/O code and has nothing to do with the Java language used. Java has a very good network performance in general; e.g. see FTP transfers in Cyberduck. Please don't write any silly annonymous postings here, please.

@cyberduck
Copy link
Collaborator Author

doml commented

Ditto.

Fugu transfers files at 11-12MBps whereas CyberDuck transfers up to about 3MBps if I'm transfering multiple items at once or ~1-1.2 MBps for a single transfer.

Also, CyberDuck crashes on large file transfers (4-20GB per file) which may or may not be a symptom of the same problem.

@cyberduck
Copy link
Collaborator Author

43a8818 commented

I concur on this one. ~88 kBps with Cduck, and ~7.8MB (yes, that's bytes) with scp.

@cyberduck
Copy link
Collaborator Author

15b0e70 commented

I'm experiencing the same problem, 240KB/s with Fugu... 20KB/s with Cyberduck...

Any idea when this is going to be fixed?

@cyberduck
Copy link
Collaborator Author

anonymous commented

I confirm it, 200KB/s with Cyberduck, 1.8MB/s by scp on terminal.

@cyberduck
Copy link
Collaborator Author

efa8001 commented

I may have just entered a duplicate of this bug at http://trac.cyberduck.ch/ticket/656

My symptoms appears to be the same. My connection is not inside a LAN, though.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

I will do a prototype replacing the current SSH implementation with Ganymed.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

The latest nightly build (2844) featueres Ganymed SSH2. Performance issues still exist, but at least uploads reach here 1MB/s over the loopback interface. This issue will only be resolved by using parallel requests as OpenSSH does. See d5ae7a7 and aff28eb.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Refer to the latest nightly builds at (http://update.cyberduck.ch/nightly/). Try Preferences > SFTP > File Transfers > Use SCP.

@cyberduck
Copy link
Collaborator Author

anonymous commented

Hi,

I'm very thankful that this is fixed however I would STRONGLY suggest that the default setting is set to SCP for SFTP transfers instead of having to dig into these forums to try to determine that this in fact solves the problem.

ALSO if someone elects SFTP as the transfer protocol it would be nice to have a message stating that there are issues with large transfers and the order of magnitude may be 10x slower as an FYI due to underlying libraries (not Cyberduck).

--Nikolaos

@cyberduck
Copy link
Collaborator Author

anonymous commented

goodness gosh this is slow. i have set it to SCP and even tiny files are taking ages to upload. the menu to select active over passive connect mode is greyed out. i think i am going to need a different FTP client unless i can fix this.

@cyberduck
Copy link
Collaborator Author

anonymous commented

Set to SCP and still it's ridiculous, 4-5MB/s vs 25-30MB/s. SFTP is dumbfounding, 200KB/s?!?

Why the fuck is this thing so slow? Thank god I'm good with a CLI; but I can't recommend this app to friends in good conscious.

@cyberduck
Copy link
Collaborator Author

p4dre commented

switch to scp ind sftp settings

solved it for me

@cyberduck
Copy link
Collaborator Author

anonymous commented

when I switched to scp, speed problem was gone.
but I can't upload files with scp mode

@cyberduck
Copy link
Collaborator Author

george commented

Just to say, I've got the same problem with the latest version. For example, downloading from a Linode VPS via SFTP today I get about 50KB/s using Cyberduck, or 800KB/s with Fugu (or rsync, for that matter).

Cyberduck is no use to me until this is fixed, so I'd certainly call that a defect rather than an enhancement. Shame, because I like its style.

@cyberduck
Copy link
Collaborator Author

Nick C commented

I'm still a little confused about whether Cyberduck supports compression over SFTP (as in rsync -z). Do you have to use either SFTP or SCP mode to get this to work?

Fugu has a checkbox for this for each connection. Even if it does, the user interface could better clarify this with a "compression" checkbox under "More Options" in the "Open Connection" window.

Of course there are times when this compression is desirable and times when it isn't, so a per-connection setting would be really handy.

@cyberduck
Copy link
Collaborator Author

popolli commented

Cyberduck was always slow and it is still in the 3.2.1. Just had 35KB/s in comparison to 650KB/s with scp.

@cyberduck
Copy link
Collaborator Author

87368a6 commented

Four years and still nothing. This is marked as an enhancement, yet this is quite a major defect. Enough so to keep me away.

@cyberduck
Copy link
Collaborator Author

9099714 commented

I'm sorry? I really like cyberduck for FTP, but marking this major defect as an enhancement makes no sense. SFTP transfers are dog slow, and nearly unusable. Why? Why no love for SFTP? Why? Don't you see all the children crying in the street for reasonable-speed encrypted transfers?

@cyberduck
Copy link
Collaborator Author

@dkocher commented

#2278 closed as duplicate.

@cyberduck
Copy link
Collaborator Author

@ylangisc commented

#4191 closed as duplicate.

@cyberduck
Copy link
Collaborator Author

d6f2f22 commented

For those complaining how SCP is faster than SFTP:
http://en.wikipedia.org/wiki/SSH_file_transfer_protocol#File_transfer_speed.2C_SCP_vs_SFTP

So, when replying to this "enhancement", please use the sftp command for comparison, not scp.

@cyberduck
Copy link
Collaborator Author

f40a067 commented

what's wrong about how filezilla does it. SFTP performance is excellent there.

@cyberduck
Copy link
Collaborator Author

74df89f commented

Hi,

Is there a way to solve these speed issues ? There's still a lot's of problem in the latest version (just downloaded the latest one today 3.5(6066) ). and sftp transfert are around 6 to 10 x slower than using the scp /sftp command line.
If I can be of any help, please let me know.

Regards,

Caeies.

@cyberduck
Copy link
Collaborator Author

psaint commented

As of Version 3.5.1 (6117) it is still an issue. Regardless of protocol selected (SFTP/SCP) transfer speeds are 10 time slower for SCP and up to 20 times slower for SFTP when compared to command line tools.

@cyberduck
Copy link
Collaborator Author

74df89f commented

I added a small patch as a POC for upload limitation. This patch will illustrate that doing parallel request for writing is faster than the current code.

However, due to my little understanding of your code, This POC is a little bit buggy since it will not clear the waiting queue for the last 64 requests ... And I've got no idea on how to do this with the current framework you're using.

Hope this helps.

Caeies

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:56 caeies]:

Back again,

For your information the bandwidth (according to download via sftp) is around 11MB/s on my LAN. So there's still a x2 factor missing somewhere.

You could try what impact the cipher has on the transfer speed. We are usually negogiating aes256-ctr wheras OpenSSH prefersaes128-ctr. You could put the weaker cipher on top in BlockCipherFactory.java for testing.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:63 caeies]:

I added a small patch as a POC for upload limitation. This patch will illustrate that doing parallel request for writing is faster than the current code.

However, due to my little understanding of your code, This POC is a little bit buggy since it will not clear the waiting queue for the last 64 requests ... And I've got no idea on how to do this with the current framework you're using.

Thanks for the patch. I have something half-baked here as well and will try to see if I can combine best of both worlds.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:62 caeies]:

Ok reading you code, I think that the main problem is that you are waiting for the ACK before continuing to send data, while openSSH just send stuff as fast as it can until he fills the queue, then check for ACK when the sent queue is filled ...

I don't see how the patch attached changes this behaviour. It will still toggle between writing and reading the status.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:65 dkocher]:

Replying to [comment:63 caeies]:

I added a small patch as a POC for upload limitation. This patch will illustrate that doing parallel request for writing is faster than the current code.

However, due to my little understanding of your code, This POC is a little bit buggy since it will not clear the waiting queue for the last 64 requests ... And I've got no idea on how to do this with the current framework you're using.

Thanks for the patch. I have something half-baked here as well and will try to see if I can combine best of both worlds.

Let me know the results with the patch attached and how it compares to the previous upload implementation.

@cyberduck
Copy link
Collaborator Author

74df89f commented

Replying to [comment:66 dkocher]:

Replying to [comment:62 caeies]:

Ok reading you code, I think that the main problem is that you are waiting for the ACK before continuing to send data, while openSSH just send stuff as fast as it can until he fills the queue, then check for ACK when the sent queue is filled ...

I don't see how the patch attached changes this behaviour. It will still toggle between writing and reading the status.

Yes, but after parallelism requests only. So it doesn't wait for the network latency just after sending the first request and so on ... it will read back the answer only after the 64th requests, so when the server starts to reply to the requests and that the requests are back to the client.

Anyway, the main problem of this patch is the lasts 64th requests that aren't checked at the end of the download.

Hope I'm clear

Caeies.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:68 caeies]:

Replying to [comment:66 dkocher]:

Replying to [comment:62 caeies]:

Ok reading you code, I think that the main problem is that you are waiting for the ACK before continuing to send data, while openSSH just send stuff as fast as it can until he fills the queue, then check for ACK when the sent queue is filled ...

I don't see how the patch attached changes this behaviour. It will still toggle between writing and reading the status.

Yes, but after parallelism requests only. So it doesn't wait for the network latency just after sending the first request and so on ... it will read back the answer only after the 64th requests, so when the server starts to reply to the requests and that the requests are back to the client.

Anyway, the main problem of this patch is the lasts 64th requests that aren't checked at the end of the download.

Hope I'm clear

Caeies.

I understand and the patch my own patch attached should fix this issue with the same characteristics of sending out multiple write requests before reading any status. Let me know about the results compared to no parallelism. Thanks again for your help here, much appreciated!

@cyberduck
Copy link
Collaborator Author

74df89f commented

Replying to [comment:67 dkocher]:

Replying to [comment:65 dkocher]:

Replying to [comment:63 caeies]:

I added a small patch as a POC for upload limitation. This patch will illustrate that doing parallel request for writing is faster than the current code.

However, due to my little understanding of your code, This POC is a little bit buggy since it will not clear the waiting queue for the last 64 requests ... And I've got no idea on how to do this with the current framework you're using.

Thanks for the patch. I have something half-baked here as well and will try to see if I can combine best of both worlds.

Let me know the results with the patch attached and how it compares to the previous upload implementation.

Sorry, I don't have the time right now to make the tests, but I will try to look at them this WE. I go quickly through your patch, and I'm not sure of all the changes you made. I will try to look carefully this WE too.

Do you have any other changes in the pipe for SFTP client ? So I don't take too much time to change something already changed :).

Thanks for your help.

Caeies.

(PS : whoops, you were quicker than me to reply :).

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:70 caeies]:

Do you have any other changes in the pipe for SFTP client ? So I don't take too much time to change something already changed :).

I attached a modified list of the preferred ciphers as a patch. You can experiment with these as well to test if that impacts throughput. I don't have anything else here.

@cyberduck
Copy link
Collaborator Author

74df89f commented

Replying to [comment:64 dkocher]:

Replying to [comment:56 caeies]:

Back again,

For your information the bandwidth (according to download via sftp) is around 11MB/s on my LAN. So there's still a x2 factor missing somewhere.

You could try what impact the cipher has on the transfer speed. We are usually negogiating aes256-ctr wheras OpenSSH prefersaes128-ctr. You could put the weaker cipher on top in BlockCipherFactory.java for testing.

So more info on this :

I try sftp with the -o Ciphers=aes256-ctr option. The download speed is around the same than previously. But the window size looks like the same as the one used by cyberduck (which looks great :). So the download speed issue is still there. I will try to switch to weaker cipher as suggested in cyberduck, but not sure it's the issue.

regards,

Caeies

@cyberduck
Copy link
Collaborator Author

74df89f commented

Replying to [comment:71 dkocher]:

Replying to [comment:70 caeies]:

Do you have any other changes in the pipe for SFTP client ? So I don't take too much time to change something already changed :).

I attached a modified list of the preferred ciphers as a patch. You can experiment with these as well to test if that impacts throughput. I don't have anything else here.
Yes there's an impact. I'm around 7MB/s now instead of around 5,2MB/s ... Still far away the ~11MB/s of the upload speed or sftp download speed. So I guess that there's still a problem in the download code. I will try to take some time to test it.

Regards,

Caeies.

PS: btw the svn wasn't able to compile this morning due to @OverRide in 3 *Sessions files.

@cyberduck
Copy link
Collaborator Author

74df89f commented

Hum, still the download speed issue. I tried a higher parallelism factor but that doesn't help. My last guess is that perhaps we are hitting performances issues in the code of the download. The window size is always "low" no more than ~98343 when the window size of sftp is higher + 134K).

Unfortunately I don't have the time to profile cyberduck and to look at where is the bottleneck. Probably in the write parts of the download requests (But that could be my computer too :). I guess that cyberduck is near its maximum performance regarding the network layout (As the upload speed seems to indicate).

I will be glad to test new code.

Regards,

Caeies.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:73 caeies]:

Replying to [comment:71 dkocher]:

Replying to [comment:70 caeies]:

Do you have any other changes in the pipe for SFTP client ? So I don't take too much time to change something already changed :).

I attached a modified list of the preferred ciphers as a patch. You can experiment with these as well to test if that impacts throughput. I don't have anything else here.
Yes there's an impact. I'm around 7MB/s now instead of around 5,2MB/s ... Still far away the ~11MB/s of the upload speed or sftp download speed. So I guess that there's still a problem in the download code. I will try to take some time to test it.

Changed priority of ciphers In dfbef4b.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:68 caeies]:

Replying to [comment:66 dkocher]:

Replying to [comment:62 caeies]:

Ok reading you code, I think that the main problem is that you are waiting for the ACK before continuing to send data, while openSSH just send stuff as fast as it can until he fills the queue, then check for ACK when the sent queue is filled ...

I don't see how the patch attached changes this behaviour. It will still toggle between writing and reading the status.

Yes, but after parallelism requests only. So it doesn't wait for the network latency just after sending the first request and so on ... it will read back the answer only after the 64th requests, so when the server starts to reply to the requests and that the requests are back to the client.

Commited upload parallelism with ACK queuing in 657339c.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:74 caeies]:

Hum, still the download speed issue. I tried a higher parallelism factor but that doesn't help. My last guess is that perhaps we are hitting performances issues in the code of the download. The window size is always "low" no more than ~98343 when the window size of sftp is higher + 134K).

Unfortunately I don't have the time to profile cyberduck and to look at where is the bottleneck. Probably in the write parts of the download requests (But that could be my computer too :). I guess that cyberduck is near its maximum performance regarding the network layout (As the upload speed seems to indicate).

I will be glad to test new code.

I'll leave this closed this until we find clear evidence of other areas for improvement.

@cyberduck
Copy link
Collaborator Author

74df89f commented

Any ideas on how to profile the java code under Mac OSX ? I tried with Shark, but it doesn't seems to be easy to profile a java application :(

@cyberduck
Copy link
Collaborator Author

@dkocher commented

Replying to [comment:78 caeies]:

Any ideas on how to profile the java code under Mac OSX ? I tried with Shark, but it doesn't seems to be easy to profile a java application :(

Your best bet is a Java Profiler which should be able to connect to the running application. jvisualvm comes with OS X but you possibly have to opt for JProfiler or YourKit Java Profiler to get anything more meaningful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants