[BIGTOP-2133] Running BPSGenerator using fat-jar fails on Mac OS X

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Component/s: blueprints
• Labels: bigpetstore
• Assignee: Kengo Seki
• Reporter: Kengo Seki
• Created: 13/Nov/15 05:33
• Updated: 29/Mar/17 18:45
• Resolved: 16/Nov/15 20:06

Description

I tried bigpetstore-mapreduce following the "Running on a hadoop cluster" section in bigtop-bigpetstore/bigpetstore-mapreduce/README.md on OS X.
First, I created fat-jar using the following command:

gradle clean shadowJar -Pfor-cluster

Then, I ran BPSGenerator, but encountered the following error:

[sekikn@mobile bigpetstore-mapreduce]$ hadoop jar build/libs/BigPetStore-1.1.0-SNAPSHOT-all.jar org.apache.bigtop.bigpetstore.generator.BPSGenerator 1000000 bigpetstore/gen
Exception in thread "main" java.io.IOException: Mkdirs failed to create /var/folders/n2/1bnspz7j4q7100jmh610zd200000gn/T/hadoop-unjar4851530034973666155/META-INF/license
	at org.apache.hadoop.util.RunJar.ensureDirectory(RunJar.java:128)
	at org.apache.hadoop.util.RunJar.unJar(RunJar.java:104)
	at org.apache.hadoop.util.RunJar.unJar(RunJar.java:81)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:209)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

This is because both META-INF/LICENSE (file) and META-INF/license (directory) exist in the fat-jar and the default filesystem of OS X doesn't differentiate them.

[sekikn@mobile bigpetstore-mapreduce]$ jar tf build/libs/BigPetStore-1.1.0-SNAPSHOT-all.jar | grep -i -E META-INF/license/?$
META-INF/LICENSE
META-INF/license/

If either of them is removed, generated jar works well.