[SPARK-5307] Add utility to help with NotSerializableException debugging

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: Spark Core
• Target Version/s: 1.3.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 18/Jan/15 07:14
• Updated: 31/Jan/15 06:42
• Resolved: 31/Jan/15 06:42

Description

Scala closures can easily capture objects unintentionally, especially with implicit arguments. I think we can do more than just relying on the users being smart about using sun.io.serialization.extendedDebugInfo to find more debug information.

Issue Links

• is duplicated by: SPARK-3694 Allow printing object graph of tasks/RDD's with a debug flag
• relates to: SPARK-3694 Allow printing object graph of tasks/RDD's with a debug flag
• links to: [Github] Pull Request #4093 (rxin)
• links to: [Github] Pull Request #4098 (rxin)
• links to: [Github] Pull Request #4297 (rxin)