Cyberduck Mountain Duck CLI

#1977 closed defect (worksforme)

Problem opening S3 buckets containing many thousands of files

Reported by: mark@… Owned by: dkocher
Priority: normal Milestone: 3.0
Component: s3 Version: 3.0b2
Severity: normal Keywords:
Cc: Architecture:
Platform:

Description

I have several S3 buckets that each contain tens of thousands of data objects. These data objects are named using the full path and filename of the original file (they are backups). For example dir1/dir2/dir3/somefile.txt

When Cyberduck opens the bucket it begins to download the file list and becomes un-responsive during this time. The refresh cannot be stopped using the stop button and the app must be force-quit.

Is it possible to prevent this by first checking the number of files in a bucket and warning the user before attempting to fetch the list? Alternately, is it possible to download the file list in stages (ie 1000 at a time)?. The last possible solution I can think of is to form a request to get only the first (pseudo) subdir. Although from my basic understanding of the S3 API, this is not possible.

Thanks for making Cyberduck!

Change History (3)

comment:1 Changed on Apr 22, 2008 at 10:04:25 PM by dkocher

Please try a nightly build from http://update.cyberduck.ch/nightly with a revision r3830 or later. Duplicate of issue #1968.

comment:2 Changed on Apr 24, 2008 at 8:38:05 PM by dkocher

  • Resolution set to worksforme
  • Status changed from new to closed

comment:3 Changed on Dec 27, 2008 at 5:08:06 PM by dkocher

  • Component changed from amazon-s3 to s3
Note: See TracTickets for help on using tickets.