Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19041

Thrift deserialization of Partition objects should intern fields

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.3.2
    • Fix Version/s: 2.4.0, 4.0.0
    • Component/s: Metastore
    • Labels:
      None

      Description

      When a client is creating large number of partitions, the thrift objects are deserialized into Partition objects. The read method of these objects does not intern the inputformat, location, outputformat which cause large number of duplicate Strings in the HMS memory. We should intern these objects while deserialization to reduce memory pressure.

        Attachments

        1. HIVE-19041.01.patch
          15 kB
          Vihang Karajgaonkar
        2. HIVE-19041.02.patch
          21 kB
          Vihang Karajgaonkar
        3. HIVE-19041.03.patch
          49 kB
          Vihang Karajgaonkar
        4. HIVE-19041.04.patch
          32 kB
          Vihang Karajgaonkar
        5. HIVE-19041.05-branch-2.patch
          24 kB
          Vihang Karajgaonkar

          Activity

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar
              Reporter:
              vihangk1 Vihang Karajgaonkar
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: