Here is a Test Plan for testing Distributed Upgrades for Block Generation Stamp
T1 Install build version 0.17 and create a new HDFS cluster of size 500 nodes.. Run random writer, sort, and then sort validation. Then shutdown cluster. Install version 0.18 build on the cluster. Start cluster with the -upgrade option.
Expected behavior: The Namenode should trigger an upgrade on all Datanodes. Then the Namenode should exit safe mode automatically. Run sort validation again to verify that data is intact.
T2 Start a DFS Upgrade and restart Namenode during the upgrade.
Expected Behaviour: Upgrade should proceed normally with appropriate messages in Namenode log
T3 Start a DFS upgrade and restart some of the Datanodes while the upgrade is occurring.
Expected behaviour: Upgrade should proceed normally with appropriate messages in Datenode logs.
T4 Start an upgrade and stop two Datanodes.
Expected Behavior: Upgrade should complete successfully because at least one replica of every block should still be available.
T4 Start an upgrade and stop half of the Datanodes.
Expected Behavior: Upgrade should stall because at least one replica of every block would not be available. Verify that useful message is displayed in the Namenode logs.
T5 Start a DFS upgrade and let it complete successfully. Then issue the admin command to finalize the upgrade.
Excepted Behavior: The Namenode should be able to finalize the cluster.
T5 Same as T4. Once the upgrade completes successfully, bring up the two Datanodes that were shut down earlier.
Expected behavior: These two Datanodes should start their own upgrades. Note that the Namenode could have already replicated these blocks.
T6 Same at T1 but with more aggressive periodic block scanner by setting dfs.datanode.scan.period.hours to 1.
Expected Behavior: Upgrade should succeed. Look at the Datanode log file to verify that no bad blocks were found by the periodic block scanner.
T7 Same as T1. Before starting the upgrade, log into a Datanode and delete a particular blk_xxx.meta file. Then start the DFS upgrade.
Expected Behavior: Look at the Namenode log to detect that the upgrade failed on that Datanode.