Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-18681

The jar package version conflict causes the task to continue to increase and grab resources

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.11.0
    • Fix Version/s: None
    • Component/s: Deployment / YARN
    • Labels:
      None

      Description

      When I submit a flink task to yarn, the default resource configuration is 1G&1core, but in fact this task will always increase resources 2core, 3core, and so on. . . 200core. . . Then I went to look at the JM log and found the following error:

      //代码占位符
      java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder; at org.apache.flink.runtime.entrypoint.parser.CommandLineOptions.<clinit>(CommandLineOptions.java:28) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_191]
      .......
      java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.runtime.entrypoint.parser.CommandLineOptionsjava.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.runtime.entrypoint.parser.CommandLineOptions at org.apache.flink.runtime.clusterframework.BootstrapTools.lambda$getDynamicPropertiesAsString$0(BootstrapTools.java:648) ~[flink-dist_2.11-1.11.1.jar:1.11.1] at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_191] at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553) ~[?:1.8.0_191] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_191]

      Finally, it is confirmed that it is caused by the commands-cli version conflict, but the task reporting error has not stopped and will continue to grab resources and increase. Is this a bug?

        Attachments

        1. dependency.log
          1.06 MB
          wangtaiyang
        2. appId.log
          1.24 MB
          wangtaiyang
        3. image-2020-07-28-15-32-51-851.png
          33 kB
          wangtaiyang
        4. yarn-hadoop-resourcemanager-x.x.x.15.log.2020-07-22-17.log
          4.20 MB
          wangtaiyang

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              apache22@163.com wangtaiyang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: