Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1623 High Availability Framework for HDFS NN
  3. HDFS-2773

HA: reading edit logs from an earlier version leaves blocks in under-construction state

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • HA branch (HDFS-1623)
    • HA branch (HDFS-1623)
    • ha, namenode
    • None

    Description

      In HDFS-2602, the code for applying OP_ADD and OP_CLOSE was changed a bit, and the new code has the following problem: if an OP_CLOSE includes new blocks (ie not previously seen in an OP_ADD) then those blocks will remain in the "under construction" state rather than being marked "complete". This is because updateBlocks always creates BlockInfoUnderConstruction regardless of the opcode. This bug only affects the upgrade path, since in trunk we always persist blocks with OP_ADDs before we call OP_CLOSE.

      Attachments

        1. hdfs-2773.txt
          9 kB
          Todd Lipcon
        2. hadoop-1.0-multiblock-file.tgz
          3 kB
          Todd Lipcon

        Activity

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: