Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-5134

FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.18.2
    • 0.18.4, 0.19.1
    • None
    • None
    • Reviewed

    Description

      From my understanding of sync/append design, an under construction block should not have any block locations associated with it in the blocksMap. So an under construction block will not be managed by ReplicationMonitor.

      However, if there is an error in the write pipeline, a lease recovery will trigger a call, commitBlockSynchronization, to NN. This call will add the successfully-recovered datanodes to blocksMap. This seems to violate the design. It should update the targets of the last block at INode instead.

      Attachments

        1. commitBlockSync3.patch
          4 kB
          Dhruba Borthakur
        2. commitBlockSync2.patch
          6 kB
          Dhruba Borthakur
        3. commitBlockSync-0.18.patch
          3 kB
          Dhruba Borthakur
        4. commitBlockSync.patch
          4 kB
          Dhruba Borthakur

        Activity

          People

            dhruba Dhruba Borthakur
            hairong Hairong Kuang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: