Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3839 Umbrella jira for Pig on Tez Performance Improvements
  3. PIG-4165

Rework Hash based aggregation for Tez

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • tez
    • None

    Description

      pig.exec.mapPartAgg (https://wiki.apache.org/pig/PigHashBasedAggInMap) is currently only used for group by if there is any aggregation to be done. In Tez, should enable hash based grouping for normal and skewed joins, groupby with or without aggregation on the map side. This will play well with the IFile optimizations done and support for writing key and list of values (TEZ-1260) and reduce the sorting time on the map side.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rohini Rohini Palaniswamy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: