Pig
  1. Pig
  2. PIG-2128

Generating the jar file takes a lot of time and is unnecessary when running Pig local mode

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.2, 0.10.0
    • Component/s: impl
    • Labels:
      None
    • Patch Info:
      Patch Available
    • Release Note:
      local mode will now skip building a jar with dependencies

      Description

      Since the Pig local mode implementation has been moved to Hadoop local it is very slow.
      We should optimize some of the steps so that it is more user friendly.
      In this case we should skip building the jar for each job as it runs in the same process.

      1. PIG-2128.patch
        1 kB
        Julien Le Dem

        Activity

        Julien Le Dem created issue -
        Julien Le Dem made changes -
        Field Original Value New Value
        Attachment PIG-2128.patch [ 12482714 ]
        Julien Le Dem made changes -
        Assignee Julien Le Dem [ julienledem ]
        Julien Le Dem made changes -
        Summary Pig local mode is very slow. Generating the jar file takes a lot of time and is unnecessary when running Pig local mode
        Description Since the Pig local mode implementation has been moved to Hadoop local it is very slow.
        We should optimize some of the steps so that it is more user friendly.
        One thing would be to skip building the jar for each job as it runs in the same process.
        Since the Pig local mode implementation has been moved to Hadoop local it is very slow.
        We should optimize some of the steps so that it is more user friendly.
        In this case we should skip building the jar for each job as it runs in the same process.
        Alan Gates made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Julien Le Dem made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note local mode will now skip building a jar with dependencies
        Fix Version/s 0.10 [ 12316246 ]
        Resolution Fixed [ 1 ]
        Dmitriy V. Ryaboy made changes -
        Fix Version/s 0.9.2 [ 12318248 ]
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Julien Le Dem
            Reporter:
            Julien Le Dem
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development