Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1203

Exception when running apex runner in non embedded mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 0.5.0
    • 0.5.0
    • runner-apex
    • None

    Description

      Tried to run the word count example application using apex runner on yarn with the command below (the command refers to local files so app when running will not be able to access these files but that is a different issue). There is an exception and the application does not get launched.

      mvn exec:java Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="-inputFile=/home/pramod/Development/test/kinglear.txt --output=/home/pramod/Development/test/wordcounts.txt --runner=ApexRunner --embeddedExecution=false" -Papex-runner

      java.nio.file.FileAlreadyExistsException: META-INF/
      at com.sun.nio.zipfs.ZipFileSystem.createDirectory(ZipFileSystem.java:422)
      at com.sun.nio.zipfs.ZipPath.createDirectory(ZipPath.java:634)
      at com.sun.nio.zipfs.ZipFileSystemProvider.createDirectory(ZipFileSystemProvider.java:201)
      at java.nio.file.Files.createDirectory(Files.java:630)
      at org.apache.beam.runners.apex.ApexYarnLauncher$2.preVisitDirectory(ApexYarnLauncher.java:266)
      at org.apache.beam.runners.apex.ApexYarnLauncher$2.preVisitDirectory(ApexYarnLauncher.java:252)
      at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:192)
      at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
      at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69)
      at java.nio.file.Files.walkFileTree(Files.java:2602)
      at java.nio.file.Files.walkFileTree(Files.java:2635)
      at org.apache.beam.runners.apex.ApexYarnLauncher.createJar(ApexYarnLauncher.java:252)
      at org.apache.beam.runners.apex.ApexYarnLauncher.launchApp(ApexYarnLauncher.java:90)
      at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:149)
      at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:63)
      at org.apache.beam.sdk.Pipeline.run(Pipeline.java:176)
      at org.apache.beam.examples.WordCount.main(WordCount.java:184)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
      at java.lang.Thread.run(Thread.java:745)

      Attachments

        Issue Links

          Activity

            People

              PramodSSImmaneni Pramod Immaneni
              PramodSSImmaneni Pramod Immaneni
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: