Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1587

Memory-based backpressure is insufficient on seek-bound workloads

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.3.0, 1.3.1, 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.8.0, 1.7.1, 1.9.0, 1.10.0, 1.10.1, 1.11.0, 1.12.0, 1.11.1
    • Fix Version/s: 1.13.0
    • Component/s: tserver

      Description

      I pushed a uniform random insert workload from a bunch of clients to the point that the vast majority of bloom filters no longer fit in buffer cache, and the compaction had fallen way behind. Thus, every inserted row turns into 40+ seeks (due to non-compact data) and takes 400-500ms. In this kind of workload, the current backpressure (based on memory usage) is insufficient to prevent ridiculously long queues.

        Attachments

        1. graph.png
          27 kB
          Todd Lipcon
        2. queue-time.png
          98 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                aserbin Alexey Serbin
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: