[SPARK-18188] Add checksum for block of broadcast

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Assignee: Davies Liu
• Reporter: Davies Liu
• Created: 31/Oct/16 23:31
• Updated: 21/Oct/22 09:24
• Resolved: 29/Nov/16 08:01

Description

着色文本There is an understanding issue for a long time: https://issues.apache.org/jira/browse/SPARK-4105, without any checksum for the blocks, it's very hard for us to identify where is the bug came from.

Shuffle blocks are compressed separate (have checksum in it), but broadcast blocks are compressed together, we should add checksum for each of separately.

Issue Links

• links to: [Github] Pull Request #15894 (davies)
• links to: [Github] Pull Request #15935 (davies)
• links to: [Github] Pull Request #21825 (10110346)
• links to: [Github] Pull Request #32385 (Ngone51)