Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11209

SNN can't checkpoint when rolling upgrade is not finalized


    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: rolling upgrades
    • Labels:


      Similar problem has been fixed with HDFS-7185. Recent change in HDFS-8432 brings this back.

      With HDFS-8432, the primary NN will not update the VERSION file to the new version after running with "rollingUpgrade" option until upgrade is finalized. This is to support more downgrade use cases.

      However, the checkpoint on the SNN is incorrectly updating the VERSION file when the rollingUpgrade is not finalized yet on the primary NN. As a result, the SNN checkpoint successfully but fail to push it to the primary NN because its version is higher than the primary NN as shown below.

      2016-12-02 05:25:31,918 ERROR namenode.SecondaryNameNode (SecondaryNameNode.java:doWork(399)) - Exception in doCheckpoint
      org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpPutFailedException: Image uploading failed, status: 403, url: http://NN:50070/imagetransfer?txid=345404754&imageFile=IMAGE&File-Le..., message: This namenode has storage info -60:221856466:1444080250181:clusterX but the secondary expected -63:221856466:1444080250181:clusterX


        1. HDFS-11209.00.patch
          7 kB
          Xiaoyu Yao
        2. HDFS-11209.01.patch
          9 kB
          Xiaoyu Yao
        3. HDFS-11209.02.patch
          10 kB
          Xiaoyu Yao
        4. HDFS-11209.03.patch
          11 kB
          Xiaoyu Yao
        5. HDFS-11209.04.patch
          10 kB
          Xiaoyu Yao

          Issue Links



              • Assignee:
                xyao Xiaoyu Yao
                xyao Xiaoyu Yao
              • Votes:
                0 Vote for this issue
                7 Start watching this issue


                • Created: