[SPARK-25885] HighlyCompressedMapStatus deserialization optimization

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Assignee: Artem Kupchinskiy
• Reporter: Artem Kupchinskiy
• Created: 30/Oct/18 12:36
• Updated: 07/Nov/18 15:12
• Resolved: 07/Nov/18 15:12

Description

HighlyCompressedMapStatus uses unnecessary indirection level during deserialization and construction. It uses ArrayBuffer, as an interim storage, before the actual map construction. Since both methods could be application hot spots under certain workloads, it is worth to get rid of that intermediate level.

Issue Links

• links to: [Github] Pull Request #22894 (Koraseg)
• links to: [Github] Pull Request #22894 (Koraseg)