Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-2216

Use a specific executor for pageSyncTimer

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.3
    • Fix Version/s: 2.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Improving throughput on paging mode is one of our concerns since our cluster uses paging a lot.

      We found that pageSyncTimer in PagingStoreImpl shared the same executor with pageCursorProvider from thread pool. In heavy load scenario like hundreds of consumers receiving messages simultaneously, it became difficult for pageSyncTimer to get the executor due to race condition. Therefore page sync was delayed and producers suffered low throughput.

       

      To achieve higher performance we assign a specific executor to pageSyncTimer to avoid racing. And we run a small-scale test on a single modified broker.

       

      Broker: 4C/8G/500G SSD

      Producer: 200 threads, non-transactional send

      Consumer 200 threads, transactional receive

      Message text size: 100-200 bytes randomly

      AddressFullPolicy: PAGE

       

      Test result:

        Only Send TPS Only Receive TPS Send&Receive TPS
      Original ver 38k 33k 3k/30k
      Modified ver 38k 34k 30k/12.5k

       

      The chart above shows that on modified broker send TPS improves from “poor” to “extremely fast”, while receive TPS drops from “extremely fast” to “not-bad” under heavy load. Considering consumer systems usually have a long processing chain after receiving messages, we don’t need too fast receive TPS. Instead, we want to guarantee send TPS to cope with traffic peak and lower producer’s delay time. Moreover, send and receive TPS in total raises from 33k to about 43k. From all above this trade-off seems beneficial and acceptable.

        Attachments

        1. contention_PR_single.svg
          570 kB
          Francesco Nigro
        2. contention_PR_global.svg
          66 kB
          Francesco Nigro
        3. contention_MASTER_global.svg
          62 kB
          Francesco Nigro

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                xuqihong Qihong Xu
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m