In my test the image file contains a data-node D0 = <name0, storageID>.
And the edits file has two record [remove D0], [add D1], where D1 = <name1, storageID>.
storageID is the same meaning that the I'm starting the same data-node on different ip addresses/ports.
I start the name-node, and I get an empty edits file and the image containing D1, which means that the
edits have been applied correctly, everything is as expected.
Then I start data-node D0 and see 2 problems that I believe are related to this issue.
1. The edits file contains 5 add/remove records in it.
There should be just 2: [remove D1], [add D0]
2. The first record in the edits file is [remove D0].
And if I try to restart the name-node it throws UnregisteredDatanodeException exception:
07/04/12 17:39:40 ERROR dfs.NameNode: org.apache.hadoop.dfs.UnregisteredDatanodeException: Data node <name0> is attempting to report storage ID DS1537505994. Node <name0> is expected to serve this storage.
at org.apache.hadoop.dfs.FSNamesystem.getDatanode(FSNamesystem.java:3461)
at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:311)
at org.apache.hadoop.dfs.FSImage.loadFSEdits(FSImage.java:672)
at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:585)
at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:220)
at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:346)
at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:251)
at org.apache.hadoop.dfs.NameNode.init(NameNode.java:173)
at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:211)
at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:820)
at org.apache.hadoop.dfs.NameNode.main(NameNode.java:828)
I tried the patch it did not fix this problem.
This patch makes sure that host2DatanodeMap is consistent with datanodeMap and thus eliminates the possibility that host2DatanodeMap has more than one instance of the datanode.