|
Suggested patch is attached. Note that that this is the first time Hadoop has a notion of versions that are not supported for upgraded. So there are a few non-trivial changes in "version upgrade" layer.. This does everything except changing hadoop-12-dfs-dir.tgz. This image will be upgraded to hadoop-14-dfs-dir.tgz. So TestDFSUpgradeFromImage fails. +1. Code looks good.
One option would be to keep the existing test and ensure that it fails. This will test that if anybody tries to upgrade an old version, then the upgrade will fail. Is it true that some codes in FSImage.loadFSImage(File) can also be removed?
Yes. It is true.. probably 2-3 checks.
Updated patch for Hudson. This has all the changes except for the changes to hadoop-12-dfs-dir.txt and hadoop-12-dfs-dir.tgz required by TestDFSUpgradeFromImage. All the Hudson tests except TestDFSUpgradeFromImage should pass.
> One option would be to keep the existing test and ensure that it fails. This will test that if anybody tries to upgrade an old version, then the upgrade will fail. In stead a new test case is added to TestDFSUpgrade.java that tests upgrade from older version. Once Hudson passes (except one test), the following extra steps are required to commit :
> All the Hudson tests except TestDFSUpgradeFromImage should pass.
One more test 'TestDistributedUpgrade' will fail too since it depends on the same DFS image. -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12381061/HADOOP-2797.patch  against trunk revision 645773. @author +1. The patch does not contain any @author tags. tests included +1. The patch appears to include 10 new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs -1. The patch appears to introduce 1 new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2338/testReport/ This message is automatically generated. More complete patch (but won't work with hudson). FindBugs warning is also fixed. I will commit this with required svn changes.
hadoop-14-dfs-dir.tgz will replace hadoop-12-dfs-dir.tgz in the trunk.
Integrated in Hadoop-trunk #475 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/475/
) |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Closed
Critical
Withdraw CRC upgrade from HDFS
This is more than just removing BlockLevelCrcs. Hadoop needs a 'LAST_UPGRADABLE_LAYOUT_VERSION' which is considered early during "version upgrade" (as opposed to distributed upgrade) because when user tries to upgrade from very old version, NameNode and DataNode should exit before making any changes to the existing diretories.