Hive
  1. Hive
  2. HIVE-538

make hive_jdbc.jar self-containing

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.3.0, 0.4.0, 0.6.0, 0.13.0
    • Fix Version/s: None
    • Component/s: JDBC
    • Labels:
      None

      Description

      Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies:
      1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro
      2. dont use hadoop configuration parameters
      3. repackage thrift and fb303 classes into hive_jdbc.jar

        Issue Links

          Activity

          Nick White made changes -
          Attachment HIVE-538.patch [ 12640663 ]
          Nick White made changes -
          Attachment HIVE-538.patch [ 12640617 ]
          Nick White made changes -
          Link This issue is related to HIVE-6593 [ HIVE-6593 ]
          Nick White made changes -
          Attachment HIVE-538.patch [ 12640617 ]
          Nick White made changes -
          Assignee Ashutosh Chauhan [ ashutoshc ] Nick White [ njw45 ]
          Nick White made changes -
          Affects Version/s 0.13.0 [ 12324986 ]
          Carl Steinbach made changes -
          Link This issue is duplicated by HIVE-3285 [ HIVE-3285 ]
          Phabricator made changes -
          Attachment HIVE-538.D2553.2.patch [ 12523001 ]
          Ashutosh Chauhan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Ashutosh Chauhan made changes -
          Assignee Ashutosh Chauhan [ ashutoshc ]
          Phabricator made changes -
          Attachment HIVE-538.D2553.1.patch [ 12520670 ]
          Carl Steinbach made changes -
          Component/s JDBC [ 12314178 ]
          Component/s Clients [ 12312587 ]
          Carl Steinbach made changes -
          Affects Version/s 0.3.1 [ 12313845 ]
          Zheng Shao made changes -
          Affects Version/s 0.6.0 [ 12314524 ]
          Affects Version/s 0.2.0 [ 12313565 ]
          Raghotham Murthy made changes -
          Description Currently, the following additional jars are required.

          * from hive/build/dist/lib
          - commons-logging-*.jar
          - hive_jdbc.jar
          - hive_metastore.jar
          - hive_serde.jar
          - hive_service.jar
          - libfb303.jar
          - libthrift.jar
          * from hadoop/build
          - hadoop-*-core.jar

          Some of the dependencies can be removed - like commons-logging, hadoop-*-core.jar. Others can be rolled in to a single jar.
          Currently, most jars in hive/build/dist/lib and the hadoop-*-core.jar are required in the classpath to run jdbc applications on hive. We need to do atleast the following to get rid of most unnecessary dependencies:
          1. get rid of dynamic serde and use a standard serialization format, maybe tab separated, json or avro
          2. dont use hadoop configuration parameters
          3. repackage thrift and fb303 classes into hive_jdbc.jar
          Raghotham Murthy made changes -
          Field Original Value New Value
          Description Currently, the following additional jars are required.

          # from hive/build/dist/lib
          # commons-logging-*.jar
          # hive_jdbc.jar
          # hive_metastore.jar
          # hive_serde.jar
          # hive_service.jar
          # libfb303.jar
          # libthrift.jar
          #
          # from hadoop/build
          # hadoop-*-core.jar

          Some of the dependencies can be removed - like commons-logging, hadoop-*-core.jar. Others can be rolled in to a single jar.
          Currently, the following additional jars are required.

          * from hive/build/dist/lib
          - commons-logging-*.jar
          - hive_jdbc.jar
          - hive_metastore.jar
          - hive_serde.jar
          - hive_service.jar
          - libfb303.jar
          - libthrift.jar
          * from hadoop/build
          - hadoop-*-core.jar

          Some of the dependencies can be removed - like commons-logging, hadoop-*-core.jar. Others can be rolled in to a single jar.
          Raghotham Murthy created issue -

            People

            • Assignee:
              Nick White
              Reporter:
              Raghotham Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development