Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-44

add Livy parameters --principal and --keytab parameters to be passed to spark-submit

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 0.2
    • None
    • Core
    • kerberized hadoop

    Description

      It seems the only correct way to run Livy server is to run it under hue user in a kerberized cluster?
      There are two advantages:
      1. hue user is normally (by default in CDH) is allowed to impersonate to other users;
      2. hue already has a keytab maintened by Cloduera Manager.

      This is explained in http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_sg_yarn_long_jobs.html
      "Configuring Spark on YARN for Long-running Applications".
      Assuming Livy Server creates long-running Spark applications.

      Without --principal and --keytab parameters (and without non-expired kerberos ticket), I am getting:

      The Spark session could not be created in the cluster: ... 37 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
      at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
      at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
      at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
      at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ...
      46 more
      Without --principal and --keytab parameters, but with a non-expired kerberos ticket for my personal principal, I am getting:

      16/01/16 22:59:00 INFO RMProxy:
      Connecting to ResourceManager at xx.yy.com/10.20.xx.yy:8032
      16/01/16 22:59:00 WARN UserGroupInformation:
      PriviledgedActionException as:rdautkha (auth:PROXY)
      via rdautkhanov@XX.COM (auth:KERBEROS)
      cause:org.apache.hadoop.security.authorize.AuthorizationException:
      User: rdautkhanov@XX.COM is not allowed to impersonate rdautkha
      ps. hue.keytab is located in

      $ sudo ls -ltr /var/run/cloudera-scm-agent/process | grep hue-KT_RENEWER | tail -1 | awk '

      {print $9}

      '
      on hosts running "Kerberos Ticket Renewer" Hue role. This is the directory that must be passed to --keytab parameter to spark-submit.

      Irrespective of Hue, Livy Server should have parameters to allow to pass --principal and --keytab to spark-submit. So it's easier (or maybe the only way) to use in kerberized environmens.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            Tagar Ruslan Dautkhanov
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment