Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-120

support hadoop map reduce in loal mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.1.0
    • None
    • None
    • Patch Available

    Description

      Currently pig support mapreduce and local as execution modes. LocalExecutionEngine is used for local and HExecutionEngine for map reduce. HExecutionEngine always expect that hadoop runs as cluster with a name node and jobtracker listing on a port.
      Though, hadoop can also run in a local mode (LocalJobRunner) this would give several advantages.
      First it would speed up the test suite significant. Second it would be possible to debug map reduce plans easily.
      For example we was able to debug and reproduce PIG-110 with this method.

      Attachments

        1. PIG-120_v_1.patch
          5 kB
          Stefan Groschupf

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. PIG-119 test suite improvements

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              joa23 Stefan Groschupf
              joa23 Stefan Groschupf
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: