Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2773

HA: reading edit logs from an earlier version leaves blocks in under-construction state

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: HA branch (HDFS-1623)
    • Fix Version/s: HA branch (HDFS-1623)
    • Component/s: ha, namenode
    • Labels:
      None

      Description

      In HDFS-2602, the code for applying OP_ADD and OP_CLOSE was changed a bit, and the new code has the following problem: if an OP_CLOSE includes new blocks (ie not previously seen in an OP_ADD) then those blocks will remain in the "under construction" state rather than being marked "complete". This is because updateBlocks always creates BlockInfoUnderConstruction regardless of the opcode. This bug only affects the upgrade path, since in trunk we always persist blocks with OP_ADDs before we call OP_CLOSE.

        Attachments

        1. hadoop-1.0-multiblock-file.tgz
          3 kB
          Todd Lipcon
        2. hdfs-2773.txt
          9 kB
          Todd Lipcon

          Activity

            People

            • Assignee:
              tlipcon Todd Lipcon
              Reporter:
              tlipcon Todd Lipcon

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment