[SPARK-11225] Prevent generate empty file

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: xukun
• Created: 21/Oct/15 02:43
• Updated: 13/Jan/16 22:12
• Resolved: 13/Jan/16 22:12

Description

If no data will be written into the bucket, it will be generate empty files. So open() must be called in the first write(key,value).

Issue Links

• is duplicated by: SPARK-7041 Avoid writing empty files in BypassMergeSortShuffleWriter
• links to: [Github] Pull Request #9191 (viper-kun)