Pig
  1. Pig
  2. PIG-2652

Skew join and order by don't trigger reducer estimation

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Fix how reducers are estimated for skew join and order operators.

      Description

      If neither PARALLEL, default parallel or mapred.reduce.tasks are set, the number of reducers is not estimated based on input size for skew joins or order by. Instead, these jobs get only 1 reducer.

      1. PIG-2652_1.patch
        0.8 kB
        Bill Graham
      2. PIG-2652_2.patch
        2 kB
        Daniel Dai
      3. PIG-2652_3_10.patch
        9 kB
        Daniel Dai
      4. PIG-2652_3.patch
        10 kB
        Daniel Dai
      5. PIG-2652_4.patch
        9 kB
        Dmitriy V. Ryaboy
      6. PIG-2652_5.patch
        11 kB
        Dmitriy V. Ryaboy
      7. PIG-2652_6.patch
        15 kB
        Dmitriy V. Ryaboy
      8. PIG-2652_7.patch
        24 kB
        Dmitriy V. Ryaboy

        Issue Links

          Activity

            People

            • Assignee:
              Dmitriy V. Ryaboy
              Reporter:
              Bill Graham
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development