Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1286

Distributed cluster upgrade



    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.13.0
    • 0.14.0
    • None
    • None


      Some data layout changes in HDFS require more than just a version upgrade introduced in HADOOP-702,
      because the cluster can function properly only when all components have upgraded, and the components
      need to communicate to each other and exchange data before they can perform the upgrade.
      The CRC upgrade discussed in HADOOP-1134 is one of such examples. Future enhancements like
      implementation of appends can change block meta-data and may require distributed upgrades.

      Distributed upgrade (DU) starts with a version upgrade (VU) so that at any time one could rollback
      all changes and start over.
      When VU is finished the name-node enters safe mode and persistently records that the DU have been started.
      It will also need to write a record when DU is finished. This is necessary to report unfinished upgrades in case
      of failure or for monitoring.

      The actual upgrade code from version vO to vN should be implemented in a separate UpgradeObject class,
      which implements interface Upgradeable.
      We create a new UpgradeObject for each pair of versions vO to vN that require a DU.
      We keep a (hard coded) table that determines which UpgradeObject(s) are applicable for the version pairs.
      Something like:

      Old version New version class names
      vO1 vN1 NameUpgradeObject1, DataUpgradeObject1
      vO2 vN2 NameUpgradeObject2, DataUpgradeObject2

      where vO1 < vN1 < vO2 < vN2 ...
      Now, if we need to upgrade from version version vX to version vY, we look for all pairs <vOi, vNi>
      in the table such that vX < vOi < vNi < vY and perform corresponding DUs one after another as they appear in the table.

      Each DU can and most probably should contain multiple UpgradeObjects.
      I'd define one object for the name-node and one for the data-nodes.
      The upgrade objects (in the same row) can communicate to each other either via existing protocols or using
      temporary protocols defined exclusively for this particular upgrade objects.
      I envision that some DUs will need to use old (vO) protocols to exchange the pre-upgrade data,
      and new (vN) protocols to reoport the upgraded data.

      UpgradeObjects should be able to bypass safe mode restrictions, be able to modify name-node data.


        1. Upgradeable.java
          2 kB
          Konstantin Shvachko
        2. DistUpgradeFramework6.patch
          66 kB
          Konstantin Shvachko

        Issue Links



              shv Konstantin Shvachko
              shv Konstantin Shvachko
              0 Vote for this issue
              1 Start watching this issue