[PHOENIX-4910] Improvements to spooled MappedByteBufferQueue files

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 5.2.1
• Assignee: Josh Elser
• Reporter: Josh Elser
• Created: 19/Sep/18 18:38
• Updated: 14/Jun/22 21:46

Description

A user ran into a JVM bug which appears to have caused a RegionServer to crash while running a topN aggregate query. This left a large number of files in /tmp after the RS had gone away (due to a JVM SIGBUS crash). MappedByteBufferQueue will buffer results in memory up to 20MB by default (controlled by phoenix.query.spoolThresholdBytes) and then start appending them to a file. I'm seeing two things which could be improved:

• If the RS exits abnormally, there is no process to clean up files - would be nice to register the deleteOnExit() hook to try to clean these up.
• There is no ability to control where MappedByteBufferQueue writes its spool file - would be nice to use something other than /tmp (I think we have a property to control this already in our config..)

FYI [e-mail]