[WICKET-6629] OOM (and disk) in AsynchronousPageStore

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 8.2.0, 9.0.0-M1
• Fix Version/s: 8.3.0, 9.0.0-M1
• Component/s: wicket
• Labels: serialization
• External issue URL: https://issues.apache.org/jira/browse/WICKET-6177
• Assignee: Martin Tzvetanov Grigorov
• Reporter: Sverre Boschman
• Created: 10/Jan/19 15:24
• Updated: 20/Jan/19 17:29
• Resolved: 11/Jan/19 12:05

Description

Symptoms:

1. Out of memory on DiskDataStore.sessionEntryMap, map contains tens of thousands of DiskDataStore$SessionEntry's
2. The vm only contains a few hundred PageStoreManager$SessionEntry's. This is expected, but not consistent with the number of DiskDataStore$SessionEntry's
3. Slowly growing disk usage, the Wicket filestore contains millions of directories, up to the point where even ls becomes unusable.

 

Snippet from our productions logs (Wicket 8.2.0):

2019-01-09 17:09:40,662 DEBUG [org.apache.wicket.session.HttpSessionStore] (default task-2513) Session unbound: 5nSIPXlkpRkGu5Oo3_wuwf801Y6TM3K1_k-95uIi
2019-01-09 17:09:40,662 DEBUG [org.apache.wicket.pageStore.DiskDataStore] (default task-2513) Removing data for pages in session with id '5nSIPXlkpRkGu5Oo3_wuwf801Y6TM3K1_k-95uIi'
2019-01-09 17:09:40,663 DEBUG [org.apache.wicket.pageStore.AsynchronousDataStore] (Wicket-AsyncDataStore-PageSavingThread) PageSavingRunnable:: Saving asynchronously: Entry [sessionId=5nSIPXlkpRkGu5Oo3_wuwf801Y6TM3K1_k-95uIi, pageId=2]...
2019-01-09 17:09:40,663 DEBUG [org.apache.wicket.pageStore.DiskDataStore] (Wicket-AsyncDataStore-PageSavingThread) Storing data for page with id '2' in session with id '5nSIPXlkpRkGu5Oo3_wuwf801Y6TM3K1_k-95uIi'

Problem (as far as I can tell)
The PageSavingRunnable of the AsynchronousPageStore saves entries to the DiskDataStore after the corresponding session has been invalidated and its DiskDataStore has been cleaned. Because the session is destroyed according to the container and Wicket session handling there is nothing triggering a cleanup for this session anymore.

 

Attached is a testcase (master branch) to showcase what is happening. Because it is a timing issue I 'faked' the testcase and temporarily stopped the PageSavingRunnable to create a scenario where the PageSavingRunnable would save an entry for an already invalidated session.