[KYLIN-2083] more RAM estimation test for MeasureAggregator and GTAggregateScanner

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v1.5.4.1
• Fix Version/s: v1.6.0
• Component/s: Tools, Build and Test
• Assignee: Dayue Gao
• Reporter: Dayue Gao
• Created: 11/Oct/16 12:29
• Updated: 26/Nov/16 08:11
• Resolved: 11/Oct/16 13:29

Description

Current RAM estimations for MeasureAggregator and GTAggregateScanner are based on test results from AggregationCacheMemSizeTest. I'd like to see if there is room for improvement, and if there is, how much we can improve.

Points I'm interested in are:

1. CompressedOops ON v.s OFF: when CompressedOops is off on large heap, each reference takes 8 bytes. I was wondering how much it will affect the RAM of AggregationCache.
2. Variable Length Aggregator: does the current estimation works well on varlen aggregator like BitmapAggregator?
3. Real Heap Usage Count via Instrumentation: the current approach to obtain the actual heap usage of objects looks fine, however, I was wondering if using Java instrumentation agent will give us a more precise number.