Hive
  1. Hive
  2. HIVE-5733

Publish hive-exec artifact without all the dependencies

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.12.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently the artifact hive-exec that is available in maven is shading all the dependencies (= the jar contains all Hive's dependencies). As other projects that are depending on Hive might be use slightly different version of the dependencies, it can easily happens that Hive's shaded version will be used instead which leads to very time consuming debugging of what is happening (for example SQOOP-1198).

      Would it be feasible publish hive-exec jar that will be build without shading any dependency? For example avro-tools is having classifier "nodeps" that represents artifact without any dependencies.

      1. HIVE-5733.1.patch
        0.8 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Hide
          Patrick Wendell added a comment -

          Hey just wanted to add a +1 and say that the current approach makes depending on Hive difficult or impossible for certain Hadoop versions due to conflicts with the protobuf library.

          Show
          Patrick Wendell added a comment - Hey just wanted to add a +1 and say that the current approach makes depending on Hive difficult or impossible for certain Hadoop versions due to conflicts with the protobuf library.
          Hide
          Amareshwari Sriramadasu added a comment -

          +1 This is much required.
          I agree it has become difficult to depend on hive exec jar, because of ql module shading all the dependencies.

          I will try to put a patch.

          Show
          Amareshwari Sriramadasu added a comment - +1 This is much required. I agree it has become difficult to depend on hive exec jar, because of ql module shading all the dependencies. I will try to put a patch.
          Hide
          Amareshwari Sriramadasu added a comment -

          As per the documentation of maven-shade-plugin the plugin will replace the project's main artifact with the shaded artifact. Giving a different classifier like nodep also was not helpful. As per the doc, the shaded artifact can be given a different name. Doc - http://maven.apache.org/plugins/maven-shade-plugin/examples/attached-artifact.html

          Attaching the patch which generates hive-exec-<version>.jar with no dependencies and hive-exec-<version>.-withdep.jar as the shaded jar.

          Show
          Amareshwari Sriramadasu added a comment - As per the documentation of maven-shade-plugin the plugin will replace the project's main artifact with the shaded artifact. Giving a different classifier like nodep also was not helpful. As per the doc, the shaded artifact can be given a different name. Doc - http://maven.apache.org/plugins/maven-shade-plugin/examples/attached-artifact.html Attaching the patch which generates hive-exec-<version>.jar with no dependencies and hive-exec-<version>.-withdep.jar as the shaded jar.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12644972/HIVE-5733.1.patch

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/200/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/200/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12644972

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12644972/HIVE-5733.1.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/200/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/200/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12644972
          Hide
          Amareshwari Sriramadasu added a comment -

          There are problems with uploaded patch, canceling

          Show
          Amareshwari Sriramadasu added a comment - There are problems with uploaded patch, canceling

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              7 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:

                Development