[BEAM-1203] Exception when running apex runner in non embedded mode

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.5.0
• Component/s: runner-apex
• Assignee: Pramod Immaneni
• Reporter: Pramod Immaneni
• Created: 22/Dec/16 01:42
• Updated: 16/May/20 13:54
• Resolved: 22/Dec/16 03:43

Description

Tried to run the word count example application using apex runner on yarn with the command below (the command refers to local files so app when running will not be able to access these files but that is a different issue). There is an exception and the application does not get launched.

mvn exec:java Dexec.mainClass=org.apache.beam.examples.WordCount -Dexec.args="-inputFile=/home/pramod/Development/test/kinglear.txt --output=/home/pramod/Development/test/wordcounts.txt --runner=ApexRunner --embeddedExecution=false" -Papex-runner

java.nio.file.FileAlreadyExistsException: META-INF/
at com.sun.nio.zipfs.ZipFileSystem.createDirectory(ZipFileSystem.java:422)
at com.sun.nio.zipfs.ZipPath.createDirectory(ZipPath.java:634)
at com.sun.nio.zipfs.ZipFileSystemProvider.createDirectory(ZipFileSystemProvider.java:201)
at java.nio.file.Files.createDirectory(Files.java:630)
at org.apache.beam.runners.apex.ApexYarnLauncher$2.preVisitDirectory(ApexYarnLauncher.java:266)
at org.apache.beam.runners.apex.ApexYarnLauncher$2.preVisitDirectory(ApexYarnLauncher.java:252)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:192)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69)
at java.nio.file.Files.walkFileTree(Files.java:2602)
at java.nio.file.Files.walkFileTree(Files.java:2635)
at org.apache.beam.runners.apex.ApexYarnLauncher.createJar(ApexYarnLauncher.java:252)
at org.apache.beam.runners.apex.ApexYarnLauncher.launchApp(ApexYarnLauncher.java:90)
at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:149)
at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:63)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:176)
at org.apache.beam.examples.WordCount.main(WordCount.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)

Issue Links

• links to: GitHub Pull Request #1684