[HDFS-8578] On upgrade, Datanode should process all storage/data dirs in parallel

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
• Component/s: datanode
• Hadoop Flags: Reviewed
• Assignee: Vinayakumar B
• Reporter: Raju Bairishetti
• Created: 11/Jun/15 10:40
• Updated: 06/Jan/17 00:52
• Resolved: 23/Feb/16 09:13

Description

Right now, during upgrades datanode is processing all the storage dirs sequentially. Assume it takes ~20 mins to process a single storage dir then datanode which has ~10 disks will take around 3hours to come up.

BlockPoolSliceStorage.java

   for (int idx = 0; idx < getNumStorageDirs(); idx++) {
      doTransition(datanode, getStorageDir(idx), nsInfo, startOpt);
      assert getCTime() == nsInfo.getCTime() 
          : "Data-node and name-node CTimes must be the same.";
    }

It would save lots of time during major upgrades if datanode process all storagedirs/disks parallelly.

Can we make datanode to process all storage dirs parallelly?

Issue Links

• blocks: HDFS-8791 block ID-based DN storage layout can be very slow for datanode on ext4
• depends upon: HDFS-9654 Code refactoring for HDFS-8578
• duplicates: HDFS-8782 Upgrade to block ID-based DN storage layout delays DN registration
• is related to: HDFS-9536 OOM errors during parallel upgrade to Block-ID based layout
• relates to: HDFS-8782 Upgrade to block ID-based DN storage layout delays DN registration