Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14350

dfs.datanode.ec.reconstruction.threads not take effect

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: datanode, ec
    • Labels:
      None

      Description

      In ErasureCodingWorker, stripedReconstructionPool is create by 

      initializeStripedBlkReconstructionThreadPool(conf.getInt(
          DFSConfigKeys.DFS_DN_EC_RECONSTRUCTION_THREADS_KEY,
          DFSConfigKeys.DFS_DN_EC_RECONSTRUCTION_THREADS_DEFAULT));
      
      private void initializeStripedBlkReconstructionThreadPool(int numThreads) {
        LOG.debug("Using striped block reconstruction; pool threads={}",
            numThreads);
        stripedReconstructionPool = DFSUtilClient.getThreadPoolExecutor(2,
            numThreads, 60, new LinkedBlockingQueue<>(),
            "StripedBlockReconstruction-", false);
        stripedReconstructionPool.allowCoreThreadTimeOut(true);
      }

      so stripedReconstructionPool is a ThreadPoolExecutor, and the queue is a LinkedBlockingQueue, then the active thread is awalys 2, the dfs.datanode.ec.reconstruction.threads not take effect.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hunhun hunshenshi
                Reporter:
                hunhun hunshenshi
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: