Details

    • Hadoop Flags:
      Reviewed

      Description

      When rolling upgrade is in progress, the standby namenode may create checkpoint. The rolling upgrade information should be added to fsimage in order to support namenode restart and continue rolling upgrade.

      1. HDFS-5945.protobuf.patch
        16 kB
        Jing Zhao
      2. h5945_20140216.patch
        13 kB
        Tsz Wo Nicholas Sze
      3. h5945_20140214.patch
        12 kB
        Tsz Wo Nicholas Sze
      4. h5945_20140213.patch
        8 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          h5945_20140213.patch: adds upgrade info to fsimage and incorporates HDFS-5905.

          Show
          Tsz Wo Nicholas Sze added a comment - h5945_20140213.patch: adds upgrade info to fsimage and incorporates HDFS-5905 .
          Hide
          Vinayakumar B added a comment -

          Hi Nicholas,
          Since HDFS-5698 is already merged to trunk, why not merge those changes first and then make changes related to this jira using protobuf itself?

          Show
          Vinayakumar B added a comment - Hi Nicholas, Since HDFS-5698 is already merged to trunk, why not merge those changes first and then make changes related to this jira using protobuf itself?
          Hide
          Vinayakumar B added a comment -

          Instead of extracting checking part to checkUpgrade() can't we use nn.getFSImage().isUpgradeFinalized() directly? This will avoid checking disks on RPC calls holding writeLock.

          Show
          Vinayakumar B added a comment - Instead of extracting checking part to checkUpgrade() can't we use nn.getFSImage().isUpgradeFinalized() directly? This will avoid checking disks on RPC calls holding writeLock.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Since HDFS-5698 is already merged to trunk, why not merge those changes first and then make changes related to this jira using protobuf itself?

          There are some conflicts with the branch. I plan to resolve them later on since I have started to work on this.

          > Instead of extracting checking part to checkUpgrade() can't we use nn.getFSImage().isUpgradeFinalized() directly? This will avoid checking disks on RPC calls holding writeLock.

          You brought up a good point about the write lock. However, isUpgradeFinalized() is different from checkUpgrade(), which is to check if upgrade is started. Since starting rolling upgrade is an admin command, we can safely assume that it won't be abused.

          Show
          Tsz Wo Nicholas Sze added a comment - > Since HDFS-5698 is already merged to trunk, why not merge those changes first and then make changes related to this jira using protobuf itself? There are some conflicts with the branch. I plan to resolve them later on since I have started to work on this. > Instead of extracting checking part to checkUpgrade() can't we use nn.getFSImage().isUpgradeFinalized() directly? This will avoid checking disks on RPC calls holding writeLock. You brought up a good point about the write lock. However, isUpgradeFinalized() is different from checkUpgrade(), which is to check if upgrade is started. Since starting rolling upgrade is an admin command, we can safely assume that it won't be abused.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h5945_20140214.patch: allows saveNamespace since checkpoint is allowed; adds new tests.

          Show
          Tsz Wo Nicholas Sze added a comment - h5945_20140214.patch: allows saveNamespace since checkpoint is allowed; adds new tests.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h5945_20140216.patch: fix a bug.

          Show
          Tsz Wo Nicholas Sze added a comment - h5945_20140216.patch: fix a bug.
          Hide
          Jing Zhao added a comment -

          Make a small change to the latest patch to use protobuf based fsimage.

          Show
          Jing Zhao added a comment - Make a small change to the latest patch to use protobuf based fsimage.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 the protobuf change looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 the protobuf change looks good.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this.

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development