Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19041

Thrift deserialization of Partition objects should intern fields

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.2, 3.0.0
    • Fix Version/s: 2.4.0, 4.0.0
    • Component/s: Metastore
    • Labels:
      None

      Description

      When a client is creating large number of partitions, the thrift objects are deserialized into Partition objects. The read method of these objects does not intern the inputformat, location, outputformat which cause large number of duplicate Strings in the HMS memory. We should intern these objects while deserialization to reduce memory pressure.

        Attachments

        1. HIVE-19041.01.patch
          15 kB
          Vihang Karajgaonkar
        2. HIVE-19041.02.patch
          21 kB
          Vihang Karajgaonkar
        3. HIVE-19041.03.patch
          49 kB
          Vihang Karajgaonkar
        4. HIVE-19041.04.patch
          32 kB
          Vihang Karajgaonkar
        5. HIVE-19041.05-branch-2.patch
          24 kB
          Vihang Karajgaonkar

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar Assign to me
              Reporter:
              vihangk1 Vihang Karajgaonkar

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment