Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4047

Break up pig withouthadoop and fat jar



    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: site
    • Fix Version/s: 0.14.0
    • Component/s: build
    • Labels:
    • Hadoop Flags:
      Incompatible change, Reviewed


      pig-withouthadoop jar is packaging pig core and pig core dependencies. But this jar should be removed due to following items:
      1. the name is confusing. User did not know what the jar is used for at a glance.
      2. it is not absolutely clear for user what the core dependencies are.
      3. it is hard to maintain dependencies, like dependencies version update. Maybe user want to try different version avro without repackaging.

      It is better to not use pig-withouthadoop jar, instead:
      1. devided without hadoop jar into pig core and pig core dependencies.
      2. save jars in 1# in lib directory
      3. in pig script, always add all the jars in lib directory into classpath and add pig core jar into classpath.
      I used pig in this way since version 0.8.1 via launching pig grunt. No issue found yet.

      Current branch-0.13 is packaging following jars into pig-withouhadoop jar:
      <include name="antlr-runtime-${antlr.version}.jar"/>
      <include name="ST4-${stringtemplate.version}.jar"/>
      <include name="jline-${jline.version}.jar"/>
      <include name="jackson-mapper-asl-${jackson.version}.jar"/>
      <include name="jackson-core-asl-${jackson.version}.jar"/>
      <include name="joda-time-${joda-time.version}.jar"/>
      <include name="guava-${guava.version}.jar"/>
      <include name="automaton-${automaton.version}.jar"/>
      <include name="jansi-${jansi.version}.jar"/>
      <include name="avro-${avro.version}.jar"/>
      <include name="avro-mapred-${avro.version}.jar"/>
      <include name="trevni-core-${avro.version}.jar"/>
      <include name="trevni-avro-${avro.version}.jar"/>
      <include name="snappy-java-${snappy. version}.jar"/>

      We could save upper jars and pig-core jar into lib directory.


        1. PIG-4047.patch
          7 kB
          fang fang chen
        2. PIG-4047-1.patch
          35 kB
          Daniel Dai
        3. PIG-4047-2.patch
          41 kB
          Daniel Dai
        4. PIG-4047-3.patch
          47 kB
          Daniel Dai
        5. PIG-4047-4.patch
          0.5 kB
          Daniel Dai

          Issue Links



              • Assignee:
                fang fang chen fang fang chen
                fang fang chen fang fang chen
              • Votes:
                0 Vote for this issue
                6 Start watching this issue


                • Created: