Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19588

Allow putting keytab file to HDFS location specified in spark.yarn.keytab

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      As a workaround for SPARK-19038 tried putting keytab in user's home directory in HDFS but this fails with

      Exception in thread "main" org.apache.spark.SparkException: Keytab file: hdfs:///user/svc_odiprd/.kt does not exist
              at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:555)
              at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:158)
              at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
              at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      

      This is yarn-client mode, so driver probably can't see HDFS while submitting a job; although I suspect it doesn't not only with yarn-client.

      Would be great to support reading keytab for kerberos ticket renewals directly from HDFS.

      We think that in some scenarios it's more secure than referencing a keytab from a local fs on a client machine that does a spark-submit.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            Tagar Ruslan Dautkhanov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment