Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4532

RPC call queue may fill due to current user lookup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
    • Fix Version/s: 0.23.7, 2.1.0-beta
    • Component/s: namenode
    • Labels:
      None

      Description

      Audit logging need to invoke UGI.getCurrentUser to log the user performing an operation. This is a synch'ed method, so audit logging is effectively serializing the completion of concurrent read ops. Under very heavy load, RPC throughput drops dramatically, the call queue fills with thousands of waiting operations, and clients begin timing out. Jstack shows all but one call handler blocked on UGI.getCurrentUser.

        Attachments

        1. HDFS-4532.patch
          19 kB
          Daryn Sharp
        2. HDFS-4532.patch
          19 kB
          Daryn Sharp
        3. HDFS-4532.branch-23.patch
          13 kB
          Daryn Sharp

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: