Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Out of memory during long/large transfers #2645

Closed
cyberduck opened this issue Oct 9, 2008 · 3 comments
Closed

Out of memory during long/large transfers #2645

cyberduck opened this issue Oct 9, 2008 · 3 comments
Assignees
@dkocher
Labels
bug core fixed
Milestone
3.1.2

Comments

@cyberduck
Copy link
Collaborator

a695180 created the issue

Transferring ~1GB of files (probably a couple hundred folders). It either stops (no error message posted) of just goes away (no message posted). Here is the last bit of the Console log, in case that helps:

log4j:WARN Continuable parsing error 31 and column 23
log4j:WARN The content of element type "log4j:configuration" must match "(renderer*,appender*,plugin*,(category|logger)*,root?,(categoryFactory|loggerFactory)?)".
Exception in thread "Background" java.lang.NullPointerException
	at ch.cyberduck.core.sftp.SFTPSession.interrupt(SFTPSession.java:303)
	at ch.cyberduck.core.sftp.SFTPPath.list(SFTPPath.java:158)
	at ch.cyberduck.core.AbstractPath.childs(AbstractPath.java:104)
	at ch.cyberduck.core.AbstractPath.childs(AbstractPath.java:91)
	at ch.cyberduck.ui.cocoa.CDInfoController$3.calculateSize(CDInfoController.java:522)
	at ch.cyberduck.ui.cocoa.CDInfoController$3.calculateSize(CDInfoController.java:523)
	at ch.cyberduck.ui.cocoa.CDInfoController$3.calculateSize(CDInfoController.java:523)
	at ch.cyberduck.ui.cocoa.CDInfoController$3.calculateSize(CDInfoController.java:523)
	at ch.cyberduck.ui.cocoa.CDInfoController$3.run(CDInfoController.java:499)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:105)
2008-10-08 18:44:13.417 Cyberduck[23036] CFURLGetFSRef failed
2008-10-08 18:44:13.506 Cyberduck[23036] CFURLGetFSRef failed
2008-10-08 18:56:37,194 [Background] ERROR ch.cyberduck.core.sftp.SFTPPath - Illegal sftp packet len: 1728053298
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
	at ch.ethz.ssh2.sftp.SFTPv3Client.receiveMessage(SFTPv3Client.java:252)
	at ch.ethz.ssh2.sftp.SFTPv3Client.scanDirectory(SFTPv3Client.java:731)
	at ch.ethz.ssh2.sftp.SFTPv3Client.ls(SFTPv3Client.java:937)
	at ch.cyberduck.core.sftp.SFTPPath.list(SFTPPath.java:104)
	at ch.cyberduck.core.AbstractPath.childs(AbstractPath.java:104)
	at ch.cyberduck.core.DownloadTransfer.childs(DownloadTransfer.java:183)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:587)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:547)
	at ch.cyberduck.core.Transfer.start(Transfer.java:643)
	at ch.cyberduck.ui.cocoa.CDTransferController$4.run(CDTransferController.java:581)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:105)
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
	at ch.ethz.ssh2.sftp.SFTPv3Client.receiveMessage(SFTPv3Client.java:252)
	at ch.ethz.ssh2.sftp.SFTPv3Client.read(SFTPv3Client.java:1491)
	at ch.cyberduck.core.sftp.SFTPInputStream.read(SFTPInputStream.java:44)
	at ch.cyberduck.core.io.ThrottledInputStream.read(ThrottledInputStream.java:72)
	at ch.cyberduck.core.Path.transfer(Path.java:553)
	at ch.cyberduck.core.Path.download(Path.java:539)
	at ch.cyberduck.core.sftp.SFTPPath.download(SFTPPath.java:449)
	at ch.cyberduck.core.Path.download(Path.java:417)
	at ch.cyberduck.core.DownloadTransfer._transferImpl(DownloadTransfer.java:337)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:466)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:552)
	at ch.cyberduck.core.Transfer.start(Transfer.java:643)
	at ch.cyberduck.ui.cocoa.CDTransferController$4.run(CDTransferController.java:581)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:105)
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
	at glguerin.io.Pathname.ensureCapacity(Pathname.java:245)
	at glguerin.io.Pathname.add(Pathname.java:355)
	at glguerin.io.PathnameFormat.putPart(PathnameFormat.java:375)
	at glguerin.io.PathnameFormat.parse(PathnameFormat.java:323)
	at glguerin.io.Pathname.setFilePath(Pathname.java:592)
	at glguerin.io.Pathname.<init>(Pathname.java:189)
	at ch.cyberduck.core.Local.init(Local.java:240)
	at ch.cyberduck.core.Local.setPath(Local.java:402)
	at ch.cyberduck.core.AbstractPath.setPath(AbstractPath.java:244)
	at ch.cyberduck.core.Local.<init>(Local.java:224)
	at ch.cyberduck.core.Local.<init>(Local.java:216)
	at ch.cyberduck.core.DownloadTransfer.childs(DownloadTransfer.java:184)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:587)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.prepare(Transfer.java:588)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:547)
	at ch.cyberduck.core.Transfer.start(Transfer.java:643)
	at ch.cyberduck.ui.cocoa.CDTransferController$4.run(CDTransferController.java:581)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:105)
2008-10-09 08:57:18,047 [Background] ERROR ch.cyberduck.core.sftp.SFTPPath - Illegal sftp packet len: 1728053301
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
	at ch.ethz.ssh2.sftp.SFTPv3Client.receiveMessage(SFTPv3Client.java:252)
	at ch.ethz.ssh2.sftp.SFTPv3Client.read(SFTPv3Client.java:1491)
	at ch.cyberduck.core.sftp.SFTPInputStream.read(SFTPInputStream.java:44)
	at ch.cyberduck.core.io.ThrottledInputStream.read(ThrottledInputStream.java:72)
	at ch.cyberduck.core.Path.transfer(Path.java:553)
	at ch.cyberduck.core.Path.download(Path.java:539)
	at ch.cyberduck.core.sftp.SFTPPath.download(SFTPPath.java:449)
	at ch.cyberduck.core.Path.download(Path.java:417)
	at ch.cyberduck.core.DownloadTransfer._transferImpl(DownloadTransfer.java:337)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:466)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:477)
	at ch.cyberduck.core.Transfer.transfer(Transfer.java:552)
	at ch.cyberduck.core.Transfer.start(Transfer.java:643)
	at ch.cyberduck.ui.cocoa.CDTransferController$4.run(CDTransferController.java:581)
	at ch.cyberduck.ui.cocoa.CDWindowController$1.run(CDWindowController.java:105)
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
Exception in thread "Background" java.lang.OutOfMemoryError: Java heap space
2008-10-09 10:24:48.769 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.913 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.913 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.915 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.916 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.917 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.918 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.919 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.921 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.922 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.923 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.924 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.925 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.926 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.927 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:24:48.929 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.731 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.901 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.902 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.903 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.904 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.906 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.907 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.908 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.909 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.910 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.911 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.913 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.914 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.915 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.916 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.917 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.919 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.920 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.921 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:26:29.922 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.407 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.529 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.530 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.531 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.532 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.534 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.535 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:27:16.536 Cyberduck[23036] CFURLGetFSRef failed
2008-10-09 10:40:56.250 Cyberduck[23036] jobjc_mapObjects() collision, objc object 4ae99770 of type (NSObject) being entered for Java object of class (ch/cyberduck/core/Local) in entry 4ae37600
2008-10-09 10:40:56.250 Cyberduck[23036] objc object 4ae99770 was entered as a NSCFDate; now is ch/cyberduck/core/Local; probable retain/release error somewhere else
2008-10-09 10:40:56.250 Cyberduck[23036] existing java object's class is (com/apple/cocoa/foundation/NSDate)
2008-10-09 10:40:56.251 Cyberduck[23036] no corresponding java entry! (did it get collected?)
ObjCJava FATAL:
jobjc_mapObjects(): mapping inconsistency -- hashtable entries are not identical
ObjCJava Exit
@cyberduck
Copy link
Collaborator Author

@dkocher commented

In a73fa62.

@cyberduck
Copy link
Collaborator Author

a695180 commented

I downloaded the newest version, and it still does not work. I ran it overnight so did not capture the logfile. It stopped after creating the folder structure, but did not transfer any data at all.

@cyberduck
Copy link
Collaborator Author

ainstushar commented

What about (http://trac.cyberduck.ch/ticket/3358)

@iterate-ch iterate-ch locked as resolved and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@dkocher dkocher

Labels
bug core fixed
Projects
None yet
Milestone
3.1.2
Development

No branches or pull requests
2 participants
@dkocher @cyberduck