Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
None
-
None
-
Reviewed
-
During a rolling upgrade from Hadoop 2.x to 3.x, NameNode cannot persist erasure coding information, and therefore a user cannot start using erasure coding feature until finalize is done.
Description
After fixing HDFS-13596, try to downgrade from 3.x to 2.x. But namenode can't start because exception occurs. The message follows
2019-01-23 17:22:18,730 ERROR org.apache.hadoop.hdfs.server.namenode.FSImage: Failed to load image from FSImageFile(file=/data1/hadoopdata/hadoop-namenode/current/fsimage_0000000000000025310, cpktTxId=0000000000
000025310)
java.lang.NullPointerException
at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.loadInternal(FSImageFormatProtobuf.java:243)
at org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf$Loader.load(FSImageFormatProtobuf.java:179)
at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:226)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:885)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:869)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:742)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:673)
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:290)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:998)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:700)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:612)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:672)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:839)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:823)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1517)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1583)
2019-01-23 17:22:19,023 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
java.io.IOException: Failed to load FSImage file, see error(s) above for more info.
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:688)
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:290)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:998)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:700)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:612)
This issue occurs because 3.x namenode saves image with EC fields during upgrade
Try to fix it
Attachments
Attachments
Issue Links
- depends upon
-
HDFS-13596 NN restart fails after RollingUpgrade from 2.x to 3.x
- Resolved
- duplicates
-
HDFS-14551 NN throws NPE if downgrade it during rolling upgrade from 3.x to 2.x
- Resolved