[OFBIZ-3558] Out of memory when importing large files /framework/datafile

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Release Branch 09.04, Release Branch 10.04, Trunk, Upcoming Branch
• Component/s: framework/webtools
• Sprint: Bug Crush Event - 21/2/2015
• Assignee: Unassigned
• Reporter: BJ Freeman
• Created: 16/Mar/10 00:16
• Updated: 11/Jan/22 08:48

Description

importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.

javax.servlet.ServletException: Servlet execution threw an exception
org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
java.util.Arrays.copyOfRange(Unknown Source)
java.lang.String.<init>(Unknown Source)
java.lang.StringBuffer.toString(Unknown Source)
java.io.StringWriter.toString(Unknown Source)
org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
org.apache.log4j.Category.callAppenders(Category.java:206)
org.apache.log4j.Category.forcedLog(Category.java:391)
org.apache.log4j.Category.log(Category.java:856)
org.ofbiz.base.util.Debug.log(Debug.java:166)
org.ofbiz.base.util.Debug.log(Debug.java:149)
org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
java.lang.reflect.Method.invoke(Unknown Source)
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
viewdatafile.run(viewdatafile.groovy:65)
org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's 
Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the 
impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many 
occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 
21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 
181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
/569/images_inventory_23320.jpg 250 250"