Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3684

ConnectionQueryServices connection leak on principal with "_HOST"

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 4.9.1, 4.10.0
    • None
    • None

    Description

      Through some internal testing, we found that Ambari's use of Phoenix to host metrics data was leaking ConnectionQueryServices (and thus HConnections and ZK connections), ultimately running into ZK's rate limiting maxClientCnxns.

      After a bit of digging around (and revisiting the old issues around this topic PHOENIX-3607, PHOENIX-3611, etc), I finally realized that the logic in ConnectionInfo was simply not correctly handling the _HOST special string in the principal (that UGI will replace with the FQDN for the current host).

      This resulted in Phoenix repeatedly re-logging in the user when they created a new Connection instead of using the UGI current user, leaking another set of connections.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            elserj Josh Elser
            arpitgupta Arpit Gupta
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment