Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-25438

Ambari starts NN with incorrect flag during upgrade

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: ambari-server
    • Labels:
      None

      Description

      Steps to reproduce:

      1. Install Ambari 2.6.x and HDP 2.6.x
      2. enable NN HA
      3. Upgrade Ambari 2.7.x
      4. Now Start HDP upgrade to 3.x and after HDFS upgrade is done pause the Upgrade
      5. Now stop both NNs are then attempt to start one from Ambari UI -  then Ambari would attempt to start the NN with "–upgrade flag"
      6. If so it fails to come up with below exception.
      2019-12-02 22:05:58,627 ERROR namenode.NameNode (NameNode.java:main(1715)) - Failed to start namenode. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not exist during upgrade. Finalize or rollback first. at org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:411) at org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:418) at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:438) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:310) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1090) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:714) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:632) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:694) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710) 2019-12-02 22:05:58,629 INFO util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not exist during upgrade. Finalize or rollback first.
      

      Ambari should check if NN upgrade is already done - if it is done already then it should not check start "upgrade" flag.

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              apappu@hortonworks.com amarnath reddy pappu
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: