Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-120

support hadoop map reduce in loal mode

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Currently pig support mapreduce and local as execution modes. LocalExecutionEngine is used for local and HExecutionEngine for map reduce. HExecutionEngine always expect that hadoop runs as cluster with a name node and jobtracker listing on a port.
      Though, hadoop can also run in a local mode (LocalJobRunner) this would give several advantages.
      First it would speed up the test suite significant. Second it would be possible to debug map reduce plans easily.
      For example we was able to debug and reproduce PIG-110 with this method.

        Attachments

        1. PIG-120_v_1.patch
          5 kB
          Stefan Groschupf

          Issue Links

            Activity

              People

              • Assignee:
                joa23 Stefan Groschupf
                Reporter:
                joa23 Stefan Groschupf
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: