[SPARK-6030] SizeEstimator gives wrong result for Integer object on 64bit JVM with UseCompressedOops on

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.2, 1.1.1, 1.2.1
• Fix Version/s: 1.4.0
• Component/s: Spark Core
• Assignee: YE
• Reporter: YE
• Due: 28/Feb/15
• Created: 26/Feb/15 06:36
• Updated: 02/May/15 22:08
• Resolved: 02/May/15 22:08

Description

Integer on 64bit JVM with UseCompressedOops on is 16bytes (verified by a related article http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html, I created a gist for that code https://gist.github.com/advancedxy/2ae7c9cc7629f3aeb679), however SizeEstimator give 24bytes for Integer.

SizeEstimator gives the wrong answer because it alignSize on internal shellSize. For Integer, there is a parent class called Number, which has zero fields. Thus the shellSize for Number is 12bytes but was aligned to 16bytes, which resulted the Integer's shellSize to be 20bytes, aligned to 24bytes.
The right path should be
1. Object-> shellSize: 12bytes, realSize: 16bytes
2. Number -> shellSize: 12bytes + 0, realSize: 16bytes
3. Integer -> shellSize: 12bytes + 4bytes(the int value), realSize: 16bytes

The fix is rather simple, I will submit a pr later.

Issue Links

• links to: [Github] Pull Request #4783 (advancedxy)