Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-7593 Supporting HSync and lease recovery
  3. HDDS-10755

[hsync] Investigate why DataNode Echo throughput is so low

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • None
    • None
    • None

    Description

      HDDS-10442 implemented a freon tool to benchmark client-DataNode echo throughput.

      The similar functionality between client-OzoneManager can reach as high as 100k echos per second. However, it is around 10k betweeen client-DataNode.

      We need to understand why.

      Attachments

        Issue Links

          is a parent of

          Sub-task - The sub-task of the issue HDDS-10908 Increase DataNode XceiverServerGrpc event loop group size

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue HDDS-10943 Freon DN Echo should skip writing payload to ratis log

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • In Progress

          Sub-task - The sub-task of the issue HDDS-10820 Freon tool DN-Echo to support GRPC and Ratis read/write mode

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-10890 [hsync] Increase default value for hdds.container.ratis.log.appender.queue.num-elements

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-10941 Add a few interesting ContainerStateMachine metrics in CSMMetrics

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-11224 Increase hdds.datanode.handler.count

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-11225 Increase ipc.server.read.threadpool.size

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDDS-10797 Remove unused UserGroupInformation object in DataNode token verifier

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDDS-10874 Freon tool DNRPCLoadGenerator should not cache client objects

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-10827 ContainerMetrics.incContainerOpsLatencies() is expensive and blocking

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-10831 Customize grpc control flow window size

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. RATIS-2079 grpc NettyServerBuilder to specify worker/boss EventLoopGroup and executor

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              weichiu Wei-Chiu Chuang
              weichiu Wei-Chiu Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: