Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14350

dfs.datanode.ec.reconstruction.threads not take effect

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      In ErasureCodingWorker, stripedReconstructionPool is create by 

      initializeStripedBlkReconstructionThreadPool(conf.getInt(
          DFSConfigKeys.DFS_DN_EC_RECONSTRUCTION_THREADS_KEY,
          DFSConfigKeys.DFS_DN_EC_RECONSTRUCTION_THREADS_DEFAULT));
      
      private void initializeStripedBlkReconstructionThreadPool(int numThreads) {
        LOG.debug("Using striped block reconstruction; pool threads={}",
            numThreads);
        stripedReconstructionPool = DFSUtilClient.getThreadPoolExecutor(2,
            numThreads, 60, new LinkedBlockingQueue<>(),
            "StripedBlockReconstruction-", false);
        stripedReconstructionPool.allowCoreThreadTimeOut(true);
      }

      so stripedReconstructionPool is a ThreadPoolExecutor, and the queue is a LinkedBlockingQueue, then the active thread is awalys 2, the dfs.datanode.ec.reconstruction.threads not take effect.

       

        Attachments

          Activity

            People

            • Assignee:
              hunhun hunshenshi
              Reporter:
              hunhun hunshenshi

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m

                  Issue deployment