[GIRAPH-1000] Multi Output support

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0, 1.1.0, 1.3.0
• Component/s: bsp, conf and scripts, graph
• Labels: features
• Assignee: Unassigned
• Reporter: Alessio Arleo
• Created: 25/Mar/15 09:54
• Updated: 12/May/20 18:16
• Resolved: 12/May/20 18:16

Description

Hadoop natively supports multiple outputs. The objective is to extend Giraph to support multiple output formats during a single giraph run.

According to the official Hadoop apidocs*, to take advantage of multiple outputs the the pattern is the following:

• Modify the job submission
• Modify the reducer class to write on the declared different outputs

Since Giraph jobs are executed as mappers, probably this approach (or at least its second part) is not feasible, so further investigation is necessary.

*https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html