Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-44

add Livy parameters --principal and --keytab parameters to be passed to spark-submit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.2
    • Fix Version/s: None
    • Component/s: Core
    • Environment:
      kerberized hadoop

      Description

      It seems the only correct way to run Livy server is to run it under hue user in a kerberized cluster?
      There are two advantages:
      1. hue user is normally (by default in CDH) is allowed to impersonate to other users;
      2. hue already has a keytab maintened by Cloduera Manager.

      This is explained in http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_sg_yarn_long_jobs.html
      "Configuring Spark on YARN for Long-running Applications".
      Assuming Livy Server creates long-running Spark applications.

      Without --principal and --keytab parameters (and without non-expired kerberos ticket), I am getting:

      The Spark session could not be created in the cluster: ... 37 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
      at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
      at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
      at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
      at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ...
      46 more
      Without --principal and --keytab parameters, but with a non-expired kerberos ticket for my personal principal, I am getting:

      16/01/16 22:59:00 INFO RMProxy:
      Connecting to ResourceManager at xx.yy.com/10.20.xx.yy:8032
      16/01/16 22:59:00 WARN UserGroupInformation:
      PriviledgedActionException as:rdautkha (auth:PROXY)
      via rdautkhanov@XX.COM (auth:KERBEROS)
      cause:org.apache.hadoop.security.authorize.AuthorizationException:
      User: rdautkhanov@XX.COM is not allowed to impersonate rdautkha
      ps. hue.keytab is located in

      $ sudo ls -ltr /var/run/cloudera-scm-agent/process | grep hue-KT_RENEWER | tail -1 | awk '

      {print $9}

      '
      on hosts running "Kerberos Ticket Renewer" Hue role. This is the directory that must be passed to --keytab parameter to spark-submit.

      Irrespective of Hue, Livy Server should have parameters to allow to pass --principal and --keytab to spark-submit. So it's easier (or maybe the only way) to use in kerberized environmens.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Tagar Ruslan Dautkhanov
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: