[IGNITE-11769] Investigate JVM crash in PDS Direct IO TeamCity suites

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: persistence
• Labels: MakeTeamcityGreenAgain
• Assignee: Unassigned
• Reporter: Ivan Rakov
• Created: 17/Apr/19 18:02
• Updated: 03/Jul/20 06:54

Description

Both PDS Direct IO suites periodically fail with JVM crash.
The issue can be reproduced on Linux machine by running IgnitePdsWithTtlTest#testTtlIsAppliedAfterRestart using ignite-direct-io classpath.
The investigation is complicated because JVM crash report is not generated during crash. After some point, JVM stays dormant for 2 minutes (waiting for JVM crash handling activities timeout) and then the process gets killed

# [ timer expired, abort... ]
Process finished with exit code 134 (interrupted by signal 6: SIGABRT

and the following error messages can be dumped to stderr before process death

corrupted double-linked list
free(): corrupted unsorted chunks

which appear to be libc error messages. Seems like Ignite corrupts virtual memory in sophisticated way which prevents normal JVM crash flow.