Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-18681

The jar package version conflict causes the task to continue to increase and grab resources

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not A Bug
    • 1.11.0
    • None
    • Deployment / YARN
    • None

    Description

      When I submit a flink task to yarn, the default resource configuration is 1G&1core, but in fact this task will always increase resources 2core, 3core, and so on. . . 200core. . . Then I went to look at the JM log and found the following error:

      //代码占位符
      java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder; at org.apache.flink.runtime.entrypoint.parser.CommandLineOptions.<clinit>(CommandLineOptions.java:28) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_191]
      .......
      java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.runtime.entrypoint.parser.CommandLineOptionsjava.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.runtime.entrypoint.parser.CommandLineOptions at org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_191] at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553) ~[?:1.8.0_191] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]

      Finally, it is confirmed that it is caused by the commands-cli version conflict, but the task reporting error has not stopped and will continue to grab resources and increase. Is this a bug?

      Attachments

        1. yarn-hadoop-resourcemanager-x.x.x.15.log.2020-07-22-17.log
          4.20 MB
          wangtaiyang
        2. image-2020-07-28-15-32-51-851.png
          33 kB
          wangtaiyang
        3. appId.log
          1.24 MB
          wangtaiyang
        4. dependency.log
          1.06 MB
          wangtaiyang

        Activity

          People

            Unassigned Unassigned
            apache22@163.com wangtaiyang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: