Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5134

FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.18.2
    • Fix Version/s: 0.18.4, 0.19.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From my understanding of sync/append design, an under construction block should not have any block locations associated with it in the blocksMap. So an under construction block will not be managed by ReplicationMonitor.

      However, if there is an error in the write pipeline, a lease recovery will trigger a call, commitBlockSynchronization, to NN. This call will add the successfully-recovered datanodes to blocksMap. This seems to violate the design. It should update the targets of the last block at INode instead.

        Attachments

        1. commitBlockSync.patch
          4 kB
          dhruba borthakur
        2. commitBlockSync2.patch
          6 kB
          dhruba borthakur
        3. commitBlockSync3.patch
          4 kB
          dhruba borthakur
        4. commitBlockSync-0.18.patch
          3 kB
          dhruba borthakur

          Activity

            People

            • Assignee:
              dhruba dhruba borthakur
              Reporter:
              hairong Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: