Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5496

ClassCastException when using Mesos HA mode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0
    • Fix Version/s: 1.2.0, 1.3.0
    • Component/s: Mesos
    • Labels:
      None

      Description

      When using the Mesos' HA mode, one cannot start the Mesos appmaster, because the following class cast exception occurs:

      java.lang.ClassCastException: org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl cannot be cast to org.apache.flink.mesos.shaded.org.apache.curator.framework.CuratorFramework
      	at org.apache.flink.mesos.util.ZooKeeperUtils.startCuratorFramework(ZooKeeperUtils.java:38)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner.createWorkerStore(MesosApplicationMasterRunner.java:510)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner.runPrivileged(MesosApplicationMasterRunner.java:320)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner$1.call(MesosApplicationMasterRunner.java:178)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner$1.call(MesosApplicationMasterRunner.java:175)
      	at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:29)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner.run(MesosApplicationMasterRunner.java:175)
      	at org.apache.flink.mesos.runtime.clusterframework.MesosApplicationMasterRunner.main(MesosApplicationMasterRunner.java:135)
      

      It seems as if the flink-mesos module relocates the curator dependency in another namespace than flink-runtime. Not sure why this is done.

        Issue Links

          Activity

          Hide
          till.rohrmann Till Rohrmann added a comment -

          The root cause seems to be that multiple modules: flink-runtime, flink-mesos and flink-kafka-0.8 relocate the curator dependency with different relocation patterns. I propose to introduce a common relocation pattern for the curator dependency in the parent pom.xml file.

          Show
          till.rohrmann Till Rohrmann added a comment - The root cause seems to be that multiple modules: flink-runtime , flink-mesos and flink-kafka-0.8 relocate the curator dependency with different relocation patterns. I propose to introduce a common relocation pattern for the curator dependency in the parent pom.xml file.
          Hide
          till.rohrmann Till Rohrmann added a comment -

          Actually, this does not work well, because we expose curator in our internal APIs, e.g. ZooKeeperStateHandleStore requires a CuratorFramework argument. Thus, relocating this code causes trouble when using the ZooKeeperStateHandleStore outside of the flink-runtime module. Ideally, we would not have to shade curator internally.

          Show
          till.rohrmann Till Rohrmann added a comment - Actually, this does not work well, because we expose curator in our internal APIs, e.g. ZooKeeperStateHandleStore requires a CuratorFramework argument. Thus, relocating this code causes trouble when using the ZooKeeperStateHandleStore outside of the flink-runtime module. Ideally, we would not have to shade curator internally.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user tillrohrmann opened a pull request:

          https://github.com/apache/flink/pull/3156

          FLINK-5496 [mesos] Relocate Mesos Protobuf dependency to avoid version conflicts

          This PR is based on #3155.

          Only relocate Mesos Protobuf dependency in flink-mesos. This avoids problems with Mesos
          because Flink pulls in Protobuf 2.5.0 via Flakka.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/tillrohrmann/flink mesosFixRelocation

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3156.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3156


          commit e68688f31ead68851a0f768a7270d8bc1b5f9ac8
          Author: Till Rohrmann <trohrmann@apache.org>
          Date: 2017-01-16T13:01:10Z

          FLINK-5496 [mesos] Relocate Mesos Protobuf dependency to avoid version conflicts

          Only relocate Mesos Protobuf dependency in flink-mesos. This avoids problems with Mesos
          because Flink pulls in Protobuf 2.5.0 via Flakka.

          commit dd2f56568745ade036de4d0ee333e7e7fdb47400
          Author: Till Rohrmann <trohrmann@apache.org>
          Date: 2017-01-16T13:14:18Z

          FLINK-5495 [mesos] Provide executor to ZooKeeperMesosWorkerStore

          The ZooKeeperMesosWorkerStore instantiates a ZooKeeperStateHandleStore which requires an
          Executor instance. This executor is now given to the ZooKeeperMesosWorkerStore.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/3156 FLINK-5496 [mesos] Relocate Mesos Protobuf dependency to avoid version conflicts This PR is based on #3155. Only relocate Mesos Protobuf dependency in flink-mesos. This avoids problems with Mesos because Flink pulls in Protobuf 2.5.0 via Flakka. You can merge this pull request into a Git repository by running: $ git pull https://github.com/tillrohrmann/flink mesosFixRelocation Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3156.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3156 commit e68688f31ead68851a0f768a7270d8bc1b5f9ac8 Author: Till Rohrmann <trohrmann@apache.org> Date: 2017-01-16T13:01:10Z FLINK-5496 [mesos] Relocate Mesos Protobuf dependency to avoid version conflicts Only relocate Mesos Protobuf dependency in flink-mesos. This avoids problems with Mesos because Flink pulls in Protobuf 2.5.0 via Flakka. commit dd2f56568745ade036de4d0ee333e7e7fdb47400 Author: Till Rohrmann <trohrmann@apache.org> Date: 2017-01-16T13:14:18Z FLINK-5495 [mesos] Provide executor to ZooKeeperMesosWorkerStore The ZooKeeperMesosWorkerStore instantiates a ZooKeeperStateHandleStore which requires an Executor instance. This executor is now given to the ZooKeeperMesosWorkerStore.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3156

          +1 to merge (I assume you've tested this)

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3156 +1 to merge (I assume you've tested this)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tillrohrmann commented on the issue:

          https://github.com/apache/flink/pull/3156

          Yes I did.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3156 Yes I did.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tillrohrmann commented on the issue:

          https://github.com/apache/flink/pull/3156

          Thanks for your review @rmetzger. Merging this PR.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3156 Thanks for your review @rmetzger. Merging this PR.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3156

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3156
          Hide
          till.rohrmann Till Rohrmann added a comment -

          1.3.0: Fixed via 04f4ecccc15f619bad00b8bd6fd727686a6b30dc
          1.2.0: Fixed via b02521262322098a7feb2a8bdb7442c21edeefd6

          Show
          till.rohrmann Till Rohrmann added a comment - 1.3.0: Fixed via 04f4ecccc15f619bad00b8bd6fd727686a6b30dc 1.2.0: Fixed via b02521262322098a7feb2a8bdb7442c21edeefd6

            People

            • Assignee:
              till.rohrmann Till Rohrmann
              Reporter:
              till.rohrmann Till Rohrmann
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development