[FLINK-1038] Adding a collection output format

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.7.0-incubating
• Assignee: Unassigned
• Reporter: Sebastian Kruse
• Created: 29/Jul/14 12:45
• Updated: 21/Sep/14 18:28
• Resolved: 21/Sep/14 18:28

Description

Similar to the existing LocalCollectionOutputFormat or Spark's collect() method, it would be nice to have a CollectionOutputFormat that also works when running jobs on a cluster. This output format gathers all results of a sink from all TaskManagers in the JVM that submitted the job plan and provides these as a collection, similar to accumulators. After all, this can help to avoid the tedious task of going to HDFS and read and parse the single result files.

PS. We have already created such an output format and can contribute it.