Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4047

Break up pig withouthadoop and fat jar

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: site
    • Fix Version/s: 0.14.0
    • Component/s: build
    • Labels:
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      pig-withouthadoop jar is packaging pig core and pig core dependencies. But this jar should be removed due to following items:
      1. the name is confusing. User did not know what the jar is used for at a glance.
      2. it is not absolutely clear for user what the core dependencies are.
      3. it is hard to maintain dependencies, like dependencies version update. Maybe user want to try different version avro without repackaging.

      It is better to not use pig-withouthadoop jar, instead:
      1. devided without hadoop jar into pig core and pig core dependencies.
      2. save jars in 1# in lib directory
      3. in pig script, always add all the jars in lib directory into classpath and add pig core jar into classpath.
      I used pig in this way since version 0.8.1 via launching pig grunt. No issue found yet.

      Current branch-0.13 is packaging following jars into pig-withouhadoop jar:
      <include name="antlr-runtime-${antlr.version}.jar"/>
      <include name="ST4-${stringtemplate.version}.jar"/>
      <include name="jline-${jline.version}.jar"/>
      <include name="jackson-mapper-asl-${jackson.version}.jar"/>
      <include name="jackson-core-asl-${jackson.version}.jar"/>
      <include name="joda-time-${joda-time.version}.jar"/>
      <include name="guava-${guava.version}.jar"/>
      <include name="automaton-${automaton.version}.jar"/>
      <include name="jansi-${jansi.version}.jar"/>
      <include name="avro-${avro.version}.jar"/>
      <include name="avro-mapred-${avro.version}.jar"/>
      <include name="trevni-core-${avro.version}.jar"/>
      <include name="trevni-avro-${avro.version}.jar"/>
      <include name="snappy-java-${snappy. version}.jar"/>

      We could save upper jars and pig-core jar into lib directory.

        Attachments

        1. PIG-4047.patch
          7 kB
          fang fang chen
        2. PIG-4047-1.patch
          35 kB
          Jianyong Dai
        3. PIG-4047-2.patch
          41 kB
          Jianyong Dai
        4. PIG-4047-3.patch
          47 kB
          Jianyong Dai
        5. PIG-4047-4.patch
          0.5 kB
          Jianyong Dai

          Issue Links

            Activity

              People

              • Assignee:
                fang fang chen fang fang chen
                Reporter:
                fang fang chen fang fang chen
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: