Uploaded image for project: 'Apache Pinot'
  1. Apache Pinot
  2. PINOT-8

Memory waste and GC slowdown in Pinot-server due to many j.l.Integer instances

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None

    Description

      I've recently analyzed a heap dump of Pinot-server with jxray (www.jxray.com). One problem that I found is that 5.8% of used heap (by byte count) and ~18% (by object count) is wasted by java.lang.Integer instances. Thus they negatively impact both memory footprint of Pinot-server and most likely its GC time as well. The GC impact is due to the fact that these objects are very numerous (nearly 1/5 of all objects), and the time to collect objects that get promoted into the Old Gen is proportional to the number of these objects. The screenshot below shows where these objects are coming: the live ones are managed by OnHeapStringDictionary._unPaddedStringToIdMap, and the garbage ones likely originate in that table as well.

      To fix this problem, it's enough to replace the java.util.HashMap above with the specialized fastutil Object2IntOpenHashMap.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            misha@cloudera.com Misha Dmitriev

            Dates

              Created:
              Updated:

              Slack

                Issue deployment