Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2888

Improve performance of POPartialAgg

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None

      Description

      During performance testing, we found that POPartialAgg can cause performance degradation for Pig jobs when the Algebraic UDFs it's being applied to aren't well suited to the operator's assumptions. Changing the implementation to a more flexible hash-based model can provide significant performance improvements.

        Attachments

        1. partialagg_patch_1.patch
          29 kB
          Dmitriy V. Ryaboy
        2. partialagg_patch_2.patch
          39 kB
          Dmitriy V. Ryaboy
        3. partialagg_patch_3.patch
          40 kB
          Dmitriy V. Ryaboy
        4. partialagg_patch_4.patch
          42 kB
          Dmitriy V. Ryaboy
        5. partialagg_patch_5.patch
          43 kB
          Dmitriy V. Ryaboy
        6. partialagg_patch_6.patch
          44 kB
          Dmitriy V. Ryaboy
        7. PIG-2888.final.patch
          97 kB
          Dmitriy V. Ryaboy

          Activity

            People

            • Assignee:
              dvryaboy Dmitriy V. Ryaboy
              Reporter:
              dvryaboy Dmitriy V. Ryaboy
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: