Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11204

flink start-scala-shell.sh do not work in security mode with kerberos authentication.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.3.3, 1.4.2, 1.5.5, 1.6.2, 1.7.0
    • Fix Version/s: None
    • Component/s: Scala Shell
    • Labels:
      None

      Description

      Hello,

      When using flink start-scala-shell.sh in a cluster with kerberos credential, the script does not supports Kerberos authentication, errors like below will occur:

      1) Fail to deploy Yarn cluster.

       

      start-scala-shell.sh yarn -n 3
      Exception in thread "main" java.lang.RuntimeException: Error deploying the YARN cluster
      at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:594)
      at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:81)
      at org.apache.flink.api.scala.FlinkShell$.deployNewYarnCluster(FlinkShell.scala:256)
      at org.apache.flink.api.scala.FlinkShell$.fetchConnectionInfo(FlinkShell.scala:165)
      at org.apache.flink.api.scala.FlinkShell$.liftedTree1$1(FlinkShell.scala:189)
      at org.apache.flink.api.scala.FlinkShell$.startShell(FlinkShell.scala:188)
      at org.apache.flink.api.scala.FlinkShell$.main(FlinkShell.scala:137)
      at org.apache.flink.api.scala.FlinkShell.main(FlinkShell.scala)
      Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm
      at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
      at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:318)
      at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:303)
      at org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:391)
      at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:385)
      at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:384)
      at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:351)
      ... 8 more
      

       

      2)Fail to fetch deployed a Flink cluster, when using the following command :

      bin/start-scala-shell.sh yarn

       

      def fetchDeployedYarnClusterInfo(
      configuration: Configuration,
      configurationDirectory: String) = {
      
      
      val args = ArrayBuffer[String](
      "-m", "yarn-cluster"
      )
      

      when fething deployed yarn cluster, with param "-m yarn-cluster" it will create new one, but has no "-n", still fail.

       
       
       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              wanger kelun wang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: