Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12734

Remove redundancy in HiveConfs serialized to UDFContext

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.1, 2.0.0, 2.2.0, 3.0.0
    • 2.2.1, 3.0.0
    • None
    • None

    Description

      HCatLoader lands up serializing one HiveConf instance per table-alias, to Pig's UDFContext. This lands up bloating the UDFContext.

      To reduce the footprint, it makes sense to serialize a default-constructed HiveConf once, and one "diff" per HCatLoader. This should reduce the time taken to kick off jobs from pig -useHCatalog scripts.

      (Note_to_self: YHIVE-540).

      Attachments

        1. HIVE-12734.1.patch
          10 kB
          Mithun Radhakrishnan
        2. HIVE-12734.2.patch
          9 kB
          Mithun Radhakrishnan
        3. HIVE-12734.2-branch-2.2.patch
          9 kB
          Mithun Radhakrishnan
        4. HIVE-12734.2-branch-2.patch
          9 kB
          Mithun Radhakrishnan

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-17621 Hive-site settings are ignored during HCatInputFormat split-calculation

          • Major - Major loss of function.
          • Closed

          Activity

            People

              cdrome Chris Drome
              mithun Mithun Radhakrishnan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: