[FLINK-11204] flink start-scala-shell.sh do not work in security mode with kerberos authentication.

• Type: Bug
• Status: Closed
• Priority: Not a Priority
• Resolution: Won't Fix
• Affects Version/s: 1.3.3, 1.4.2, 1.5.5, 1.6.2, 1.7.0
• Component/s: Scala Shell
• Assignee: Unassigned
• Reporter: wangge
• Created: 19/Dec/18 13:09
• Updated: 24/Sep/21 10:02
• Resolved: 24/Sep/21 10:02

Description

Hello,

When using flink start-scala-shell.sh in a cluster with kerberos credential, the script does not supports Kerberos authentication, errors like below will occur:

1) Fail to deploy Yarn cluster.

 

start-scala-shell.sh yarn -n 3
Exception in thread "main" java.lang.RuntimeException: Error deploying the YARN cluster
at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:594)
at org.apache.flink.yarn.cli.FlinkYarnSessionCli.createCluster(FlinkYarnSessionCli.java:81)
at org.apache.flink.api.scala.FlinkShell$.deployNewYarnCluster(FlinkShell.scala:256)
at org.apache.flink.api.scala.FlinkShell$.fetchConnectionInfo(FlinkShell.scala:165)
at org.apache.flink.api.scala.FlinkShell$.liftedTree1$1(FlinkShell.scala:189)
at org.apache.flink.api.scala.FlinkShell$.startShell(FlinkShell.scala:188)
at org.apache.flink.api.scala.FlinkShell$.main(FlinkShell.scala:137)
at org.apache.flink.api.scala.FlinkShell.main(FlinkShell.scala)
Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:318)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:303)
at org.apache.hadoop.security.UserGroupInformation.isAuthenticationMethodEnabled(UserGroupInformation.java:391)
at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:385)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:384)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:351)
... 8 more

 

2)Fail to fetch deployed a Flink cluster, when using the following command :

bin/start-scala-shell.sh yarn

 

def fetchDeployedYarnClusterInfo(
configuration: Configuration,
configurationDirectory: String) = {


val args = ArrayBuffer[String](
"-m", "yarn-cluster"
)

when fething deployed yarn cluster, with param "-m yarn-cluster" it will create new one, but has no "-n", still fail.