Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4532

RPC call queue may fill due to current user lookup

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0
    • Fix Version/s: 0.23.7, 2.1.0-beta
    • Component/s: namenode
    • Labels:
      None

      Description

      Audit logging need to invoke UGI.getCurrentUser to log the user performing an operation. This is a synch'ed method, so audit logging is effectively serializing the completion of concurrent read ops. Under very heavy load, RPC throughput drops dramatically, the call queue fills with thousands of waiting operations, and clients begin timing out. Jstack shows all but one call handler blocked on UGI.getCurrentUser.

      1. HDFS-4532.branch-23.patch
        13 kB
        Daryn Sharp
      2. HDFS-4532.patch
        19 kB
        Daryn Sharp
      3. HDFS-4532.patch
        19 kB
        Daryn Sharp

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development