Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6800

Support Datanode layout changes with rolling upgrade

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We need to handle attempts to rolling-upgrade the DataNode to a new storage directory layout.

      One approach is to disallow such upgrades. If we choose this approach, we should make sure that the system administrator gets a helpful error message and a clean failure when trying to use rolling upgrade to a version that doesn't support it. Based on the compatibility guarantees described in HDFS-5535, this would mean that any future DataNode layout changes would require a major version upgrade.

      Another approach would be to support rolling upgrade from an old DN storage layout to a new layout. This approach requires us to change our documentation to explain to users that they should supply the -rollback command on the command-line when re-starting the DataNodes during rolling rollback. Currently the documentation just says to restart the DataNode normally.

      Another issue here is that the DataNode's usage message describes rollback options that no longer exist. The help text says that the DN supports -rollingupgrade rollback, but this option was removed by HDFS-6005.

        Attachments

        1. HDFS-6800.patch
          3 kB
          James Thomas
        2. HDFS-6800.2.patch
          2 kB
          James Thomas
        3. HDFS-6800.3.patch
          3 kB
          James Thomas
        4. HDFS-6800.4.patch
          3 kB
          James Thomas
        5. HDFS-6800.5.patch
          3 kB
          James Thomas
        6. HDFS-6800.6.patch
          5 kB
          James Thomas
        7. HDFS-6800.7.patch
          6 kB
          James Thomas
        8. HDFS-6800.8.patch
          4 kB
          James Thomas

          Issue Links

            Activity

              People

              • Assignee:
                james.thomas James Thomas
                Reporter:
                cmccabe Colin McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: