Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable upload folder with 14300 files via SFTP (Java Out of memory problem) #3358

Closed
cyberduck opened this issue Jul 22, 2009 · 3 comments
Closed

Unable upload folder with 14300 files via SFTP (Java Out of memory problem) #3358

cyberduck opened this issue Jul 22, 2009 · 3 comments
Assignees
@dkocher
Labels
bug core high priority worksforme
Milestone
3.3.1

Comments

@cyberduck
Copy link
Collaborator

a4f70d8 created the issue

I can't upload folder contained only 14300 files.
Upload queue display:

    1. Get list of folder several seconds
    1. Transfer not finished

In console log appear:

22.7.09 12:42:03 Cyberduck[52011] ERROR: Attempt to set dropItem=(null), dropChildIndex=20 when [-1 - 0] is valid range of indices.
22.7.09 12:42:03 Cyberduck[52011] ERROR: Attempt to set dropItem=(null), dropChildIndex=20 when [-1 - 0] is valid range of indices. 
22.7.09 12:41:57 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:42:20 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:42:51 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:43:42 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:45:52 Cyberduck[52011] *** Assertion failure in -[NSProgressIndicator lockFocus], /SourceCache/AppKit/AppKit-949.46/AppKit.subproj/NSView.m:4755 
22.7.09 12:45:52 Cyberduck[52011] Exception -[NSProgressIndicator(0x1666d930) lockFocus] failed with window=0x15a9c470, windowNumber=62845, [self isHiddenOrHasHiddenAncestor]=1 raised during heart beat. Ignoring.... 
22.7.09 12:46:34 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:57:10 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space 
22.7.09 12:57:49 [0x0-0x9ae9ae].ch.sudo.cyberduck[52011] Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space

Workaround with limitation:

Compress (zip) folder via Finder -> upload -> and uncompress (unzip)
@cyberduck
Copy link
Collaborator Author

ainstushar commented

Same problem here:

Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
	at java.util.HashMap.resize(HashMap.java:508)
	at java.util.HashMap.addEntry(HashMap.java:799)
	at java.util.HashMap.put(HashMap.java:431)
	at ch.cyberduck.core.Transfer.exists(Transfer.java:396)
	at ch.cyberduck.core.SyncTransfer.compare(SyncTransfer.java:326)
	at ch.cyberduck.core.SyncTransfer.isSelectable(SyncTransfer.java:212)
	at ch.cyberduck.core.Transfer.isIncluded(Transfer.java:438)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:601)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:613)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:573)
	at ch.cyberduck.core.Transfer.start(Transfer.java:671)
	at ch.cyberduck.ui.cocoa.CDTransferController$4.run(CDTransferController.java:581)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:77)

@cyberduck
Copy link
Collaborator Author

a695180 commented

This does not suprize me. My previous bug #2645: Out of memory during long/large transfers was reported fixed, but when I tried it again after the reported fix, it failed again. I forgot to report that.

@cyberduck
Copy link
Collaborator Author

@dkocher commented

As of 3.3 we no longer keep references to all files in memory. This should be fixed. Please reopen if still an issue.

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug core high priority worksforme
Projects
None yet
Milestone
3.3.1
Development

No branches or pull requests
2 participants
@dkocher @cyberduck