Uploaded image for project: 'Giraph'
  1. Giraph
  2. GIRAPH-1000

Multi Output support

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0, 1.1.0, 1.3.0
    • Fix Version/s: None
    • Component/s: bsp, conf and scripts, graph
    • Labels:

      Description

      Hadoop natively supports multiple outputs. The objective is to extend Giraph to support multiple output formats during a single giraph run.

      According to the official Hadoop apidocs*, to take advantage of multiple outputs the the pattern is the following:

      • Modify the job submission
      • Modify the reducer class to write on the declared different outputs

      Since Giraph jobs are executed as mappers, probably this approach (or at least its second part) is not feasible, so further investigation is necessary.

      *https://hadoop.apache.org/docs/r1.2.1/api/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.html

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Alessio Alessio Arleo
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: