[FLINK-11088] Allow pre-install Kerberos authentication keytab discovery on YARN

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.11.0
• Component/s: Deployment / YARN
• Labels: pull-request-available
• Assignee: Rong Rong
• Reporter: Rong Rong
• Created: 06/Dec/18 23:48
• Updated: 13/Mar/20 15:15
• Resolved: 13/Mar/20 15:15

Description

Currently flink-yarn assumes keytab is shipped as application master environment local resource on client side and will be distributed to all the TMs. This does not work for YARN proxy user mode [1] since proxy user or super user might not have access to actual users' keytab, but can request delegation tokens on users' behalf.

Based on the type of security options for long-living YARN service[2], we propose to have the keytab file path discovery configurable depending on the launch mode of the YARN client.

Reference:
[1] https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html
[2] https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html#Securing_Long-lived_YARN_Services

Issue Links

• links to: GitHub Pull Request #7702