Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5013

Flink Kinesis connector doesn't work on old EMR versions

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.4
    • Component/s: Kinesis Connector
    • Labels:
      None

      Description

      A user reported on the mailing list that our Kinesis connector doesn't work with EMR 4.4.0: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kinesis-Connector-Dependency-Problems-td9790.html

      The problem seems to be that Flink is loading older libraries from the "YARN container classpath", which on EMR contains the default Amazon libraries.

      We should try to shade kinesis and its amazon dependencies into a different namespace.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tzulitai Tzu-Li (Gordon) Tai
                Reporter:
                rmetzger Robert Metzger
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: