Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1897

GSSAPI negotiation single-threaded and very slow under concurrency

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.3.0
    • 1.3.0
    • rpc, security
    • None

    Description

      I'm running an Impala workload with ~10 concurrent clients submitting queries that take a few seconds each on a 9 node cluster. The master is pegging on core in negotiation, and has a bunch of other negotiation threads blocked on a mutex within SASL.

      Each negotiation is taking 2-3 seconds due to waiting on this lock.

      This severely restricts connection throughput, and probably needs to be addressed for 1.3.

      Attachments

        1. rcache.png
          17 kB
          Todd Lipcon

        Issue Links

          causes

          Sub-task - The sub-task of the issue IMPALA-6432 Default rpc_negotiation_timeout_ms may cause queries to fail on large clusters

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. KUDU-1907 Lock contention in SASL Kerberos negotiation

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              danburkert Dan Burkert
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: