Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: regionserver
    • Labels:
      None

      Description

      In the context of HBASE-10169, we're adding the ability to batch Coprocessor endpoint calls per regionserver, using the same batching that happens in the RegionServer.multi() calls. However, execution of each of the calls will still happen serially on each RegionServer. For Coprocessor endpoint calls, it might help to parallelize these, since each execution could be of indeterminate length.

      Since it may help to parallelize the Coprocessor endpoint invocations, it raises the question of whether other operations handled in multi() calls should also be parallelized, or should we just rely on macro-scale parallelization through the RPC handler threads?

        Issue Links

          Activity

          Gary Helmling created issue -
          Gary Helmling made changes -
          Field Original Value New Value
          Link This issue is related to HBASE-10169 [ HBASE-10169 ]
          Hide
          Andrew Purtell added a comment -

          Since it may help to parallelize the Coprocessor endpoint invocations, it raises the question of whether other operations handled in multi() calls should also be parallelized, or should we just rely on macro-scale parallelization through the RPC handler threads?

          Do we want parallel endpoint invocation executing in RPC and tying up the handlers? What if the number of target regions on the server is ~100 or even ~1000?

          Show
          Andrew Purtell added a comment - Since it may help to parallelize the Coprocessor endpoint invocations, it raises the question of whether other operations handled in multi() calls should also be parallelized, or should we just rely on macro-scale parallelization through the RPC handler threads? Do we want parallel endpoint invocation executing in RPC and tying up the handlers? What if the number of target regions on the server is ~100 or even ~1000?
          Hide
          Andrew Purtell added a comment -

          Ping Jingcheng Du - your thoughts?

          Show
          Andrew Purtell added a comment - Ping Jingcheng Du - your thoughts?
          Andrew Purtell made changes -
          Parent HBASE-10171 [ 12684883 ]
          Issue Type Improvement [ 4 ] Sub-task [ 7 ]
          Hide
          Enis Soztutar added a comment -

          I think we should decouple more query handling and RPC handling. If there are threads lying around, we would like to handle multi gets in parallel to reduce the latency. This would encourage more batching for gets on the client side, and less number of RPC's in turn.

          Show
          Enis Soztutar added a comment - I think we should decouple more query handling and RPC handling. If there are threads lying around, we would like to handle multi gets in parallel to reduce the latency. This would encourage more batching for gets on the client side, and less number of RPC's in turn.
          Hide
          Jingcheng Du added a comment -

          I have the same concerns with Andrew Purtell. I think it's better to have a separate pool with a limited size for the parallel endpoint execution.

          Show
          Jingcheng Du added a comment - I have the same concerns with Andrew Purtell . I think it's better to have a separate pool with a limited size for the parallel endpoint execution.

            People

            • Assignee:
              Unassigned
              Reporter:
              Gary Helmling
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development