[HDFS-14367] EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0
• Component/s: ec
• Hadoop Flags: Reviewed
• Assignee: Guo Lei
• Reporter: Guo Lei
• Created: 13/Mar/19 03:45
• Updated: 10/Jun/21 07:44
• Resolved: 12/May/20 13:22

Description

The workQueue length wasn't specified, so the thread number never be increase.

The thread number increase to maximumPoolSize only when the workQueue is full.

file location: org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java

private void initializeStripedBlkReconstructionThreadPool(int numThreads) {
  LOG.debug("Using striped block reconstruction; pool threads={}",
      numThreads);
  stripedReconstructionPool = DFSUtilClient.getThreadPoolExecutor(2,
      numThreads, 60, new LinkedBlockingQueue<>(),
      "StripedBlockReconstruction-", false);
  stripedReconstructionPool.allowCoreThreadTimeOut(true);
}

Issue Links

• is duplicated by: HDFS-14350 dfs.datanode.ec.reconstruction.threads not take effect