[SPARK-20148] Extend the file commit interface to allow subscribing to task commit messages

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.0
• Component/s: SQL
• Assignee: Eric Liang
• Reporter: Eric Liang
• Created: 29/Mar/17 23:28
• Updated: 30/Mar/17 03:59
• Resolved: 30/Mar/17 03:59

Description

The internal FileCommitProtocol interface returns all task commit messages in bulk to the implementation when a job finishes. However, it is sometimes useful to access those messages before the job completes, so that the driver gets incremental progress updates before the job finishes.

Issue Links

• links to: [Github] Pull Request #17475 (ericl)