Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2133

Running BPSGenerator using fat-jar fails on Mac OS X

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: blueprints
    • Labels:

      Description

      I tried bigpetstore-mapreduce following the "Running on a hadoop cluster" section in bigtop-bigpetstore/bigpetstore-mapreduce/README.md on OS X.
      First, I created fat-jar using the following command:

      gradle clean shadowJar -Pfor-cluster
      

      Then, I ran BPSGenerator, but encountered the following error:

      [sekikn@mobile bigpetstore-mapreduce]$ hadoop jar build/libs/BigPetStore-1.1.0-SNAPSHOT-all.jar org.apache.bigtop.bigpetstore.generator.BPSGenerator 1000000 bigpetstore/gen
      Exception in thread "main" java.io.IOException: Mkdirs failed to create /var/folders/n2/1bnspz7j4q7100jmh610zd200000gn/T/hadoop-unjar4851530034973666155/META-INF/license
      	at org.apache.hadoop.util.RunJar.ensureDirectory(RunJar.java:128)
      	at org.apache.hadoop.util.RunJar.unJar(RunJar.java:104)
      	at org.apache.hadoop.util.RunJar.unJar(RunJar.java:81)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:209)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
      

      This is because both META-INF/LICENSE (file) and META-INF/license (directory) exist in the fat-jar and the default filesystem of OS X doesn't differentiate them.

      [sekikn@mobile bigpetstore-mapreduce]$ jar tf build/libs/BigPetStore-1.1.0-SNAPSHOT-all.jar | grep -i -E META-INF/license/?$
      META-INF/LICENSE
      META-INF/license/
      

      If either of them is removed, generated jar works well.

        Attachments

        1. BIGTOP-2133.patch
          0.7 kB
          Kengo Seki
        2. BIGTOP-2133.01.patch
          1.0 kB
          Kengo Seki

          Activity

            People

            • Assignee:
              sekikn Kengo Seki
              Reporter:
              sekikn Kengo Seki
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: