Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8444

Analysis perf regression after IMPALA-7616

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 3.2.0
    • Fix Version/s: Impala 3.3.0
    • Component/s: Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-2

      Description

      The patch for IMPALA-7616 caused a performance regression in analysis time when run in an environment with ~1k roles and 10.5k. privileges. The regression is evident when run as a role that has a large number of privileges.

      Following is the stack to look for when jstacking the coordnator.

      "Thread-21" #49 prio=5 os_prio=0 tid=0x000000000cd6e000 nid=0x6a3d runnable [0x00007fa28e4a0000]
         java.lang.Thread.State: RUNNABLE
              at java.lang.String.toLowerCase(String.java:2670)
              at org.apache.impala.catalog.PrincipalPrivilege.buildPrivilegeName(PrincipalPrivilege.java:82)
              at org.apache.impala.catalog.PrincipalPrivilege.getName(PrincipalPrivilege.java:143)
              at org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:423)
              - locked <0x00007fa376987100> (a org.apache.impala.catalog.AuthorizationPolicy)
              at org.apache.impala.catalog.AuthorizationPolicy.listPrivileges(AuthorizationPolicy.java:443)
              - locked <0x00007fa376987100> (a org.apache.impala.catalog.AuthorizationPolicy)
              at org.apache.sentry.provider.cache.SimpleCacheProviderBackend.getPrivileges(SimpleCacheProviderBackend.java:75)
              at org.apache.sentry.policy.db.SimpleDBPolicyEngine.getPrivileges(SimpleDBPolicyEngine.java:98)
              at org.apache.sentry.provider.common.ResourceAuthorizationProvider.getPrivileges(ResourceAuthorizationProvider.java:147)
              at org.apache.sentry.provider.common.ResourceAuthorizationProvider.doHasAccess(ResourceAuthorizationProvider.java:120)
              at org.apache.sentry.provider.common.ResourceAuthorizationProvider.hasAccess(ResourceAuthorizationProvider.java:107)
              at org.apache.impala.authorization.AuthorizationChecker.hasAccess(AuthorizationChecker.java:215)
              at org.apache.impala.authorization.AuthorizationChecker.checkAccess(AuthorizationChecker.java:128)
              at org.apache.impala.analysis.AnalysisContext.authorizePrivilegeRequest(AnalysisContext.java:592)
              at org.apache.impala.analysis.AnalysisContext.authorize(AnalysisContext.java:564)
              at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:415)
              at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1240)
              at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1210)
              at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1182)
              at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:158)
      

      Issue worsens when running concurrent workloads, because the underlying Sentry listPrivileges() is synchronized and that serializes all the query analysis requests.

        public synchronized Set<String> listPrivileges(Set<String> groups,  <====
            ActiveRoleSet roleSet) {
          Set<String> privileges = Sets.newHashSet();
          if (roleSet != ActiveRoleSet.ALL) {
            throw new UnsupportedOperationException("Impala does not support role subsets.");
          }
      

      Notes:

      • If the authorization metadata footprint is small, this issue can be ignored.
      • One workaround is to run the query using a role that has very small number of privileges (revoke privileges that are not necessary to run a given query).
      • Another workaround is to disable Sentry authorization.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                fredyw Fredy Wijaya
                Reporter:
                bharathv bharath v
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: