Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12734

Remove redundancy in HiveConfs serialized to UDFContext

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.1, 2.0.0, 2.2.0, 3.0.0
    • Fix Version/s: 2.2.1, 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      HCatLoader lands up serializing one HiveConf instance per table-alias, to Pig's UDFContext. This lands up bloating the UDFContext.

      To reduce the footprint, it makes sense to serialize a default-constructed HiveConf once, and one "diff" per HCatLoader. This should reduce the time taken to kick off jobs from pig -useHCatalog scripts.

      (Note_to_self: YHIVE-540).

        Attachments

        1. HIVE-12734.1.patch
          10 kB
          Mithun Radhakrishnan
        2. HIVE-12734.2.patch
          9 kB
          Mithun Radhakrishnan
        3. HIVE-12734.2-branch-2.2.patch
          9 kB
          Mithun Radhakrishnan
        4. HIVE-12734.2-branch-2.patch
          9 kB
          Mithun Radhakrishnan

          Issue Links

            Activity

              People

              • Assignee:
                cdrome Chris Drome
                Reporter:
                mithun Mithun Radhakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: