Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5138

Support HDFS upgrade in HA

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.1.1-beta
    • 2.4.0
    • None
    • None
    • Incompatible change, Reviewed

    Description

      With HA enabled, NN wo't start with "-upgrade". Since there has been a layout version change between 2.0.x and 2.1.x, starting NN in upgrade mode was necessary when deploying 2.1.x to an existing 2.0.x cluster. But the only way to get around this was to disable HA and upgrade.

      The NN and the cluster cannot be flipped back to HA until the upgrade is finalized. If HA is disabled only on NN for layout upgrade and HA is turned back on without involving DNs, things will work, but finaliizeUpgrade won't work (the NN is in HA and it cannot be in upgrade mode) and DN's upgrade snapshots won't get removed.

      We will need a different ways of doing layout upgrade and upgrade snapshot. I am marking this as a 2.1.1-beta blocker based on feedback from others. If there is a reasonable workaround that does not increase maintenance window greatly, we can lower its priority from blocker to critical.

      Attachments

        1. HDFS-5138.branch-2.001.patch
          179 kB
          Jing Zhao
        2. HDFS-5138.patch
          185 kB
          Aaron Myers
        3. HDFS-5138.patch
          184 kB
          Aaron Myers
        4. HDFS-5138.patch
          192 kB
          Aaron Myers
        5. HDFS-5138.patch
          191 kB
          Aaron Myers
        6. HDFS-5138.patch
          181 kB
          Aaron Myers
        7. HDFS-5138.patch
          181 kB
          Aaron Myers
        8. HDFS-5138.patch
          177 kB
          Aaron Myers
        9. HDFS-5138.patch
          171 kB
          Aaron Myers
        10. HDFS-5138.patch
          140 kB
          Aaron Myers
        11. HDFS-5138.patch
          140 kB
          Aaron Myers
        12. HDFS-5138.patch
          138 kB
          Aaron Myers
        13. hdfs-5138-branch-2.txt
          185 kB
          Todd Lipcon

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8127 NameNode Failover during HA upgrade can cause DataNode to finalize upgrade

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8469 Lockfiles are not being created for datanode storage directories

          • Major - Major loss of function.
          • Patch Available
          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3958 Integrate upgrade/finalize/rollback with external journals

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-3133 Add support for DFS upgrade with HA enabled

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HDFS-2952 HA: NN should not start with upgrade option or with a pending an unfinalized upgrade

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-3225 Revist upgrade snapshots, roll back, finalize to enable rolling upgrades

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8241 Remove unused NameNode startup option -finalize

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-5223 Allow edit log/fsimage format changes without changing layout version

          • Major - Major loss of function.
          • Open
          requires

          Bug - A problem which impairs or prevents the functions of the product. HDFS-5840 Follow-up to HDFS-5138 to improve error handling during partial upgrade failures

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              atm Aaron Myers
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: