[FLINK-17729] Make mandatory to have lib/, plugin/ and dist in yarn.provided.lib.dirs

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Client / Job Submission, Deployment / YARN
• Labels: pull-request-available
• Assignee: Kostas Kloudas
• Reporter: Kostas Kloudas
• Created: 15/May/20 11:59
• Updated: 15/May/20 18:13
• Resolved: 15/May/20 18:13

Description

This will make mandatory for users of the "yarn.provided.lib.dirs" to also include the lib/, plugin/ and flink-dist jar. If these are not included in the shared resources, then the feature cannot be used and all the dependencies of an application are expected to be shipped from the client (as it was the previous behaviour). If they are provided, then these are going to be used and NOT what the user may have locally (e.g. different flink or log4j versions).

The reason for this requirement is to avoid unpleasant surprises with classloading issues.

Issue Links

• links to: GitHub Pull Request #12170