MaxMemory needs increasing for large document sets: Memory Leak with XMLFileModule

• Key: FOR-591
• Type: Bug
• Status: Open
• Priority: Blocker
• Assignee: Unassigned
• Reporter: Ross Gardler
• Component/s: Core operations
• Affects Version/s: 0.7, 0.8
• Fix Version/s: 0.9-dev

Description

Since the docs restructuring for the 0.7 release it has become necessary to increase the maxmemory to be able to build the Forrest site. We gained three copies of the documents so suddenly have a large document set to trigger memory leakage issues.

Possibly COCOON-1574 "Memory Leak with XMLFileModule".

Does someone have the tools to run some diagnostics?

(NB maxmemory has been increased in our site-author/forrest.properties, if we resolve this issue it should be reduced again)

Comments

[David Crossley]

I tried going back to the default again today, still neeed.

[Ron Blaschke]

This is a Cocoon issue.
http://issues.apache.org/bugzilla/show_bug.cgi?id=36162

[Ross Gardler]

Cocoon has now moved to JIra, the issue tracker for this is now:

http://issues.apache.org/jira/secure/ViewIssue.jspa?key=COCOON-1574

[Tim Williams]

I think I have recently compounded this issue with similar simple caching in the locationmap -- although it will likely be *much*less noticable (strings vs doms). I'll try to figure out what the "official store" for caching means and how to properly make use of it for caching.

[David Crossley]

Just an update note ... there has been some activity towards fixing this at COCOON-1574

[Gavin]

Another update note :

I have just added a comment (to COCOON-1574) to see how they are getting on, if no near time frame for resolving it then maybe this will have to be put back to a 0.9 release.

Ralph Goers has replied and said will look at it in the next few days (http://issues.apache.org/jira/browse/COCOON-1574#action_12371833)

[David Crossley]

Setting Urgency=urgent. This issue was already present in the 0.7 release, so it probably isn't a blocker. It became apparent due to our hefty site-author docs arrangement, so it is a blocker for that situation. It would certainly be good to have fixed.

[David Crossley]

This recent thread at cocoon-dev continued to address the issues:
Re: questions on store usage
http://marc.theaimsgroup.com/?t=113977732400002

[Antonio Gallardo]

Are we able to update cocoon if I wrote a simple fix for that?

[Ross Gardler]

I am unaware of any reason not to try an upgrade Cocoon. However, I have no idea how to build Cocoon anymore (since the move to Maven). If you know how - that would be really cool.

[David Crossley]

Changed the Issue Summary and Description.

[Tim Williams]

Move to the next release. We need Cocoon to address the dependent issue first.

Discussed:
http://marc.theaimsgroup.com/?t=115188914900001&r=1&w=2

[David Crossley]

Ralph Goers at Cocoon has implemented the new XPathXMLFileModule to address the issues with XMLFileModule. Currently in Cocoon-2.1 branch, soon to be also in trunk.

See recent comments in COCOON-1574.

[Tim Williams]

Can anyone confirm that this isn't a "drop-in" replacement? A simple switch from XMLFileModule to XPathXMLFileModule leads me to an error like:

"Caused by: org.apache.avalon.framework.configuration.ConfigurationException: No attribute named "src" is associated with the configuration element "file" at <generated>file:/Users/twilliams/Development/forrest/main/webapp/WEB-INF/cocoon.xconf:248:56"

My initial hunch is that the original version allowed lazy loading of the source the new XPathXMLFileModule does not but before I looked into it too much I wanted to see if someone experiences the same?

fwiw, tracking just this little problem down in the unnavigable mess that has become the cocoon source leads me wanting to pursue the oft proposed forrest2...

[David Crossley]

Further discussion today:
  Re: XPathXMLFileModule differences
  http://thread.gmane.org/gmane.text.xml.forrest.devel/27275

See abovementioned comment 2007-Dec-27 here and at COCOON-1574.