[FLINK-17641] How to secure flink applications on yarn on multi-tenant environment

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Deployment / YARN
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Ethan Li
• Created: 12/May/20 18:03
• Updated: 09/Dec/21 10:43

Description

This is a question I wish to get some insights on.

We are trying to support and secure flink on shared yarn cluster. Besides the security provided by yarn side (queueACL, kerberos), what I noticed is that flink CLI can still interact with the flink job as long as it knows the jobmanager rpc port/hostname and rest.port, which can be obtained easily with yarn command.

Also on the UI side, on yarn cluster, users can visit flink job UI via yarn proxy using browser. As long as the user can authenticate and view yarn resourcemanager webpage, he/she can visit the flink UI without any problem. This basically means Flink UI is wide-open to corp internal users.

On the internal connection side, I am aware of the support added in 1.10 to limit the mTLS connection by configuring security.ssl.internal.cert.fingerprint (https://ci.apache.org/projects/flink/flink-docs-stable/ops/security-ssl.html)

This works but it is not very flexible. Users need to update the config if the cert changes before they submit a new job.

I asked the similar question on the mailing list before. I am really interested in how other folks deal with this issue. Thanks.

Issue Links

• relates to: FLINK-18787 Add a "cluster-info" command to the Flink command line client