[HDFS-15080] Fix the issue in reading persistent memory cached data with an offset

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.1.4, 3.2.2
• Component/s: caching, datanode
• Assignee: Feilong He
• Reporter: Feilong He
• Created: 25/Dec/19 03:17
• Updated: 08/Jan/20 09:11
• Resolved: 08/Jan/20 09:06

Description

Some applications can read a segment of pmem cache with an offset specified. The previous implementation for pmem cache read with DirectByteBuffer didn't cover this situation.

Let me explain further. In our test, we used spark SQL to run some TPC-DS workload to read the cache data and hits read exception. This was due to the missed seek offset arg, which is used in spark SQL to read data packet by packet.

Issue Links

• is related to: HDFS-13762 Support non-volatile storage class memory(SCM) in HDFS cache directives