Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10066

Hive on Tez job submission through WebHCat doesn't ship Tez artifacts

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.2.0
    • Component/s: Tez, WebHCat
    • Labels:

      Description

      From Hitesh Shah:
      "Tez is a client-side only component ( no daemons, etc ) and therefore it is meant to be installed on the gateway box ( or where its client libraries are needed by any other services’ daemons). It does not have any cluster dependencies both in terms of libraries/jars as well as configs. When it runs on a worker node, everything was pre-packaged and made available to the worker node via the distributed cache via the client code. Hence, its client-side configs are also only needed on the same (client) node as where it is installed. The only other install step needed is to have the tez tarball be uploaded to HDFS and the config has an entry “tez.lib.uris” which points to the HDFS path. "

      We need a way to pass client jars and tez-site.xml to the LaunchMapper.

      We should create a general purpose mechanism here which can supply additional artifacts per job type.

        Attachments

        1. HIVE-10066.patch
          16 kB
          Eugene Koifman
        2. HIVE-10066.3.patch
          15 kB
          Eugene Koifman
        3. HIVE-10066.2.patch
          16 kB
          Eugene Koifman

          Issue Links

            Activity

              People

              • Assignee:
                ekoifman Eugene Koifman
                Reporter:
                ekoifman Eugene Koifman
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: