Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17042

Jackson version conflict for Flink 1.9

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Cannot Reproduce
    • 1.9.0
    • None
    • BuildSystem / Shaded
    • None

    Description

      From the user mailing list:

      Hi,

      After I upgrade flink 1.9, I got the following error message on EMR, it works locally on IntelliJ.

      I'm explicitly declaring the dependency as
      implementation 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.10.1'
      and I have
      implementation group: 'com.amazonaws', name: 'aws-java-sdk-emr', version: '1.11.595'

      java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
             at com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
             at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
             at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
             at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
             at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107)
             at com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
             at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
             at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
             at com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
             at com.coinbase.util.KmsClient$.getSnowflakeUsernamePassword(KmsClient.scala:21)
             at com.coinbase.ml.RunFlinkJob$.runBatch(RunFlinkJob.scala:94)
             at com.coinbase.ml.RunFlinkJob$.runFlinkJob(RunFlinkJob.scala:38)
             at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint$.main(CmdLineParser.scala:76)
             at com.coinbase.ml.FlinkFeatureProcessingJobEntryPoint.main(CmdLineParser.scala)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
             at java.lang.reflect.Method.invoke(Method.java:498)
             at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
             at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
             at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
             at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
             at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
             at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
             at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
             at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
             at java.security.AccessController.doPrivileged(Native Method)
             at javax.security.auth.Subject.doAs(Subject.java:422)
             at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
             at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
             at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
      

      The outputs for "jar -tf flink-dist_2.11-1.9.0.jar | grep jackson.databind.ObjectMapper"

      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper$1.class
      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper$2.class
      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper$3.class
      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper$DefaultTypeResolverBuilder.class
      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper$DefaultTyping.class
      org/apache/flink/shaded/jackson2/com/fasterxml/jackson/databind/ObjectMapper.class
      org/apache/flink/mesos/shaded/com/fasterxml/jackson/databind/ObjectMapper$1.class
      org/apache/flink/mesos/shaded/com/fasterxml/jackson/databind/ObjectMapper$2.class
      org/apache/flink/mesos/shaded/com/fasterxml/jackson/databind/ObjectMapper$DefaultTypeResolverBuilder.class
      org/apache/flink/mesos/shaded/com/fasterxml/jackson/databind/ObjectMapper$DefaultTyping.class
      org/apache/flink/mesos/shaded/com/fasterxml/jackson/databind/ObjectMapper.class
      com/fasterxml/jackson/databind/ObjectMapper$1.class
      com/fasterxml/jackson/databind/ObjectMapper$2.class
      com/fasterxml/jackson/databind/ObjectMapper$DefaultTypeResolverBuilder.class
      com/fasterxml/jackson/databind/ObjectMapper$DefaultTyping.class
      com/fasterxml/jackson/databind/ObjectMapper.class
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            danny0405 Danny Chen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment