[FLINK-5364] Rework JAAS configuration to support user-supplied entries

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0, 1.3.0
• Component/s: Runtime / Coordination
• Labels: kerberos security
• Assignee: Eron Wright
• Reporter: Eron Wright
• Created: 19/Dec/16 00:50
• Updated: 28/Feb/19 14:12
• Resolved: 11/Jan/17 20:21

Description

Recent issues (see linked) have brought to light a critical deficiency in the handling of JAAS configuration.

1. the MapR distribution relies on an explicit JAAS conf, rather than in-memory conf used by stock Hadoop.
2. the ZK/Kafka/Hadoop security configuration is supposed to be independent (one can enable each element separately) but isn't.

Perhaps we should rework the JAAS conf code to merge any user-supplied configuration with our defaults, rather than using an all-or-nothing approach.

We should also address some recent regressions:

1. The HadoopSecurityContext should be installed regardless of auth mode, to login with UserGroupInformation, which:

• handles the HADOOP_USER_NAME variable.
• installs an OS-specific user principal (from UnixLoginModule etc.) unrelated to Kerberos.
• picks up the HDFS/HBASE delegation tokens.

2. Fix the use of alternative authentication methods - delegation tokens and Kerberos ticket cache.

Issue Links

• relates to: FLINK-5055 Security feature crashes JM for certain Hadoop versions even though using no Kerberos
• relates to: FLINK-5379 Flink CliFrontend does not return when not logged in with kerberos
• relates to: FLINK-5350 Don't overwrite existing Jaas config property
• relates to: FLINK-5361 Flink shouldn't require Kerberos credentials
• links to: GitHub Pull Request #3057