Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
Description
I've recently analyzed a heap dump of Pinot-server with jxray (www.jxray.com). One problem that I found is that 5.8% of used heap (by byte count) and ~18% (by object count) is wasted by java.lang.Integer instances. Thus they negatively impact both memory footprint of Pinot-server and most likely its GC time as well. The GC impact is due to the fact that these objects are very numerous (nearly 1/5 of all objects), and the time to collect objects that get promoted into the Old Gen is proportional to the number of these objects. The screenshot below shows where these objects are coming: the live ones are managed by OnHeapStringDictionary._unPaddedStringToIdMap, and the garbage ones likely originate in that table as well.
To fix this problem, it's enough to replace the java.util.HashMap above with the specialized fastutil Object2IntOpenHashMap.
