Pig
  1. Pig
  2. PIG-204

Revision 643583 for PIG-94 broke input splits

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.0.0
    • Fix Version/s: 0.1.0
    • Component/s: impl
    • Labels:
      None

      Description

      For revisions of the code after 643583, all pig jobs run in 1 map, instead of multiple maps. The jobs still produce correct results, but they do not run in parallel.

      1. PIG-204_1_20080411.patch
        4 kB
        Arun C Murthy
      2. PIG-204_0_20080410.patch
        0.9 kB
        Arun C Murthy

        Activity

        Alan Gates created issue -
        Hide
        Arun C Murthy added a comment -

        Fix to PigInputFormat...

        Show
        Arun C Murthy added a comment - Fix to PigInputFormat...
        Arun C Murthy made changes -
        Field Original Value New Value
        Attachment PIG-204_0_20080410.patch [ 12379895 ]
        Hide
        Arun C Murthy added a comment -

        Updated patch, I had to fix MapreducePlanCompile.connectInputs to correctly propagate the properties, also fixed EvalSpec.copy to do the same. Finally PigInputFormat.getSplits now uses the correct default value for pig.input.splittable.

        Show
        Arun C Murthy added a comment - Updated patch, I had to fix MapreducePlanCompile.connectInputs to correctly propagate the properties, also fixed EvalSpec.copy to do the same. Finally PigInputFormat.getSplits now uses the correct default value for pig.input.splittable .
        Arun C Murthy made changes -
        Attachment PIG-204_1_20080411.patch [ 12379955 ]
        Hide
        Arun C Murthy added a comment -

        Submitting patch for review, passes all unit-tests locally.

        Show
        Arun C Murthy added a comment - Submitting patch for review, passes all unit-tests locally.
        Arun C Murthy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Alan Gates added a comment -

        Fix checked in at revision 647356. Thanks Arun for your quick response.

        Show
        Alan Gates added a comment - Fix checked in at revision 647356. Thanks Arun for your quick response.
        Alan Gates made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.1.0 [ 12312848 ]
        Resolution Fixed [ 1 ]
        Alan Gates made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Alan Gates
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development