Hive
  1. Hive
  2. HIVE-4562

HIVE-3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: Build Infrastructure
    • Labels:
      None

      Description

      Some jars of Hive are required not only by the client but also the server (every Hadoop slave),
      though we could use 'add jar' command to add all the jars in dis-cache ,
      but in common way ,we may add these jars in $HADOOP_HOME/lib/ of every salve of the Hadoop Cluster,
      and need restart all the tasktrackers .

      For example:
      When using hive stats, If we use mysql as tmp stats db ,every salve of the Hadoop Cluster should contain
      mysql-connector-java-****.jar in $HADOOP_HOME/lib/

      And for column stats
      In all slaves $HADOOP_HOME/lib/ should contain:
      jackson-core-asl-1.8.8.jar
      jackson-jaxrs-1.8.8.jar
      jackson-mapper-asl-1.8.8.jar
      jackson-xc-1.8.8.jar

      These jars should be separated from other common client-side-jars .

      1. HIVE-4562-1.patch
        2 kB
        caofangkun
      2. HIVE-4562-2.patch
        2 kB
        caofangkun

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Ashutosh Chauhan added a comment - This issue has been fixed and released as part of 0.12 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #2126 (See https://builds.apache.org/job/Hive-trunk-h0.21/2126/)
          HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) (Revision 1488744)

          Result = FAILURE
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488744
          Files :

          • /hive/trunk/ql/build.xml
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #2126 (See https://builds.apache.org/job/Hive-trunk-h0.21/2126/ ) HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) (Revision 1488744) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488744 Files : /hive/trunk/ql/build.xml
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #223 (See https://builds.apache.org/job/Hive-trunk-hadoop2/223/)
          HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) (Revision 1488744)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488744
          Files :

          • /hive/trunk/ql/build.xml
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #223 (See https://builds.apache.org/job/Hive-trunk-hadoop2/223/ ) HIVE-4562 : HIVE3393 brought in Jackson library,and these four jars should be packed into hive-exec.jar (caofangkun via Ashutosh Chauhan) (Revision 1488744) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488744 Files : /hive/trunk/ql/build.xml
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, caofangkun!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, caofangkun!
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Edward Capriolo added a comment -

          I just upgraded to hive 0.10 and I had to but these in the auxlib.

          We REALLY need to start using jar jar here. The bigger hive-exec gets the more painful its gets to build udfs. In particular if a udf or input format uses jackson the target library needs to match the version. This is already a problem because things like apache-commons get built into hive-exec . So if ANY other project you are trying to use leverages commons you are forced to use hive's version in some cases.

          Show
          Edward Capriolo added a comment - I just upgraded to hive 0.10 and I had to but these in the auxlib. We REALLY need to start using jar jar here. The bigger hive-exec gets the more painful its gets to build udfs. In particular if a udf or input format uses jackson the target library needs to match the version. This is already a problem because things like apache-commons get built into hive-exec . So if ANY other project you are trying to use leverages commons you are forced to use hive's version in some cases.
          Hide
          caofangkun added a comment -

          Only jackson-core-asl-1.8.8.jar
          and jackson-mapper-asl-1.8.8.jar
          need to be packed in.

          Show
          caofangkun added a comment - Only jackson-core-asl-1.8.8.jar and jackson-mapper-asl-1.8.8.jar need to be packed in.
          Hide
          caofangkun added a comment -

          Add patch

          Show
          caofangkun added a comment - Add patch
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Yes, I think a repack is necessary.

          Show
          Zhuoluo (Clark) Yang added a comment - Yes, I think a repack is necessary.
          Hide
          caofangkun added a comment -

          HIVE-3393 brought in Jackson library, but not repack the new four jackson*.jar into hive-exec-*.jar

          Show
          caofangkun added a comment - HIVE-3393 brought in Jackson library, but not repack the new four jackson*.jar into hive-exec-*.jar
          Hide
          Zhuoluo (Clark) Yang added a comment -

          I think the trunk version has no such problems.
          trunk has unzipped this jars and repack them into hive-exec.jar.
          caofangkun IMHO, you can use "grep -nC5 unzip ql/build.xml" to look into the logic.
          And you can put mysql-jdbc-connector.jar in the HIVE_AUX_JARS_PATH.
          For these reasons, shall we mark this issue "Won't Fix"?

          Show
          Zhuoluo (Clark) Yang added a comment - I think the trunk version has no such problems. trunk has unzipped this jars and repack them into hive-exec.jar. caofangkun IMHO, you can use "grep -nC5 unzip ql/build.xml" to look into the logic. And you can put mysql-jdbc-connector.jar in the HIVE_AUX_JARS_PATH. For these reasons, shall we mark this issue "Won't Fix"?

            People

            • Assignee:
              caofangkun
              Reporter:
              caofangkun
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development