[HDFS-14872] Read HDFS Blocks in Random Order - ASF JIRA

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 2.8.5, 3.2.1
Fix Version/s: None
Component/s: hdfs-client
Labels:
None

Description

When the HDFS client is downloading (copying) an entire file, allow the client to download the blocks in random order. If a lot of clients are reading the same file, in parallel, they will all download the first block, the second block, and so on, stampeding down the line.

It would be interesting to spread the load across across all the available DataNodes.