Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11708

Positional read will fail if replicas moved to different DNs after stream is opened

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Scenario:
      1. File was written to DN1, DN2 with RF=2
      2. File stream opened to read and kept. Block Locations are [DN1,DN2]
      3. One of the replica (DN2) moved to another datanode (DN3) due to datanode dead/balancing/etc.
      4. Latest block locations in NameNode will be DN1 and DN3 in the 'same order'
      5. DN1 went down, but not yet detected as dead in NameNode.
      6. Client start reading using positional read api "read(pos, buf[], offset, length)"

      Attachments

        1. HDFS-11708-01.patch
          12 kB
          Vinayakumar B
        2. HDFS-11708-02.patch
          16 kB
          Vinayakumar B
        3. HDFS-11708-03.patch
          16 kB
          Vinayakumar B
        4. HDFS-11708-04.patch
          16 kB
          Vinayakumar B
        5. HDFS-11708-05.patch
          16 kB
          Vinayakumar B
        6. HDFS-11708-07.patch
          22 kB
          Vinayakumar B
        7. HDFS-11708-07.patch
          22 kB
          Vinayakumar B
        8. HDFS-11708-branch-2.7-07.patch
          21 kB
          Vinayakumar B
        9. HDFS-11708-branch-2.8-07.patch
          22 kB
          Vinayakumar B
        10. HDFS-11708-branch-2-07.patch
          22 kB
          Vinayakumar B
        11. HDFS-11708-HDFS-11898-06.patch
          26 kB
          Vinayakumar B

        Issue Links

          Activity

            People

              vinayakumarb Vinayakumar B
              vinayakumarb Vinayakumar B
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: