[FLINK-5013] Flink Kinesis connector doesn't work on old EMR versions

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.1.4
• Component/s: Connectors / Kinesis
• Assignee: Tzu-Li (Gordon) Tai
• Reporter: Robert Metzger
• Created: 04/Nov/16 09:57
• Updated: 28/Nov/16 13:00
• Resolved: 28/Nov/16 13:00

Description

A user reported on the mailing list that our Kinesis connector doesn't work with EMR 4.4.0: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kinesis-Connector-Dependency-Problems-td9790.html

The problem seems to be that Flink is loading older libraries from the "YARN container classpath", which on EMR contains the default Amazon libraries.

We should try to shade kinesis and its amazon dependencies into a different namespace.

Issue Links

• links to: GitHub Pull Request #2787