[IMPALA-8521] Lots of "unreleased ByteBuffers allocated by read()" errors from HDFS client

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: Backend
• Epic Color: ghx-label-7
• Assignee: Sahil Takiar
• Reporter: Tim Armstrong
• Created: 07/May/19 23:16
• Updated: 07/Sep/23 22:18

Description

I'm looking at some job logs and seeing a bunch of errors like this. I don't know if it's benign or if it's something more serious.

I0507 07:34:53.934693 20195 scan-range.cc:607] dd4d6eb8d2ad9587:6b44fe1b00000002] Cache read failed for scan range: file=hdfs://localhost:20500/test-warehouse/f861f1a3/nation.tbl disk_id=0 offset=1024  exclusive_hdfs_fh=0xec09220 num_remote_bytes=0 cancel_status= buffer_queue=0 num_buffers_in_readers=0 unused_iomgr_buffers=0 unused_iomgr_buffer_bytes=0 blocked_on_buffer=0. Switching to disk read path.
W0507 07:34:53.934787 20195 DFSInputStream.java:668] dd4d6eb8d2ad9587:6b44fe1b00000002] closing file /test-warehouse/f861f1a3/nation.tbl, but there are still unreleased ByteBuffers allocated by read().  Please release java.nio.DirectByteBufferR[pos=1024 lim=2048 cap=2199].