Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14872

Read HDFS Blocks in Random Order

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.8.5, 3.2.1
    • None
    • hdfs-client
    • None

    Description

      When the HDFS client is downloading (copying) an entire file, allow the client to download the blocks in random order. If a lot of clients are reading the same file, in parallel, they will all download the first block, the second block, and so on, stampeding down the line.

      It would be interesting to spread the load across across all the available DataNodes.

      Attachments

        Activity

          People

            Unassigned Unassigned
            belugabehr David Mollitor
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: