[BIGTOP-3701] Remove deprecated Hadoop fs-image file after uninstalling HDFS-namenode

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: build, hadoop
• Assignee: Yuqi Gu
• Reporter: Yuqi Gu
• Created: 08/Jun/22 09:27
• Updated: 23/Jun/22 04:13
• Resolved: 15/Jun/22 08:38

Description

Usecase scenario:

When deploying HDFS, Yarn and MapReduce services by Ambari-Mpack from Bigtop RPMs,

firstly I installed Hadoop-2.10 from Bigtop-1.5 repo, then I uninstalled it and re-installed Hadoop 3.2 from Bigtop-3.1.0 repo.

It would fail to format namenode:

ERROR namenode.NameNode (NameNode.java:main(1795)) - Failed to start namenode.
java.io.IOException:
File system image contains an old layout version -63.
An upgrade to version -65 is required.
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:280)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1135)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:750)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:658)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:734)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:977)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:950)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1723)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1790)
2022-05-24 15:26:35,431 INFO  util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: java.io.IOException:
File system image contains an old layout version -63.
An upgrade to version -65 is required.

The user have to restart NameNode with the "-rollingUpgrade started" option if a rolling upgrade is already started; or restart NameNode with the "-upgrade" option to start a new upgrade. It would confuse some new Hadoop users.

 

It's better to remove deprecated Hadoop fs-image file after uninstalling HDFS-namenode in RPM spec.