[HDFS-16520] Improve EC pread: avoid potential reading whole block

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1, 3.3.2
• Fix Version/s: 3.4.0, 3.3.5
• Component/s: dfsclient, ec, erasure-coding
• Labels: pull-request-available
• Assignee: daimin
• Reporter: daimin
• Due: 24/Mar/22
• Created: 24/Mar/22 07:03
• Updated: 06/May/22 18:23
• Resolved: 06/May/22 18:23

Description

HDFS client 'pread' represents 'position read', this kind of read just need a range of data instead of reading the whole file/block. By using BlockReaderFactory#setLength, client tells datanode the block length to be read from disk and sent to client.
To EC file, the block length to read is not well set, by default using 'block.getBlockSize() - offsetInBlock' to both pread and sread. Thus datanode read much more data and send to client, and abort when client closes connection. There is a lot waste of resource to this situation.

Issue Links

• links to: GitHub Pull Request #4104