Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4958

Tez autoparallelism estimation for order by is higher than mapreduce

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 0.18.0
    • None
    • None

    Description

      The input size is calculated from the size of the samples in memory. Size in memory is usually 4x or more than the serialized size. Mapreduce estimates the number of reducers based on serialized size.

      Attachments

        1. PIG-4958-1.patch
          38 kB
          Rohini Palaniswamy
        2. PIG-4958-2.patch
          35 kB
          Rohini Palaniswamy
        3. PIG-4958-withoutsecurity.patch
          13 kB
          Rohini Palaniswamy

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. TEZ-3385 DAGClient API should be accessible outside of DAG submission

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PIG-4959 Tez autoparallelism estimation for skewed join is higher than mapreduce

          • Major - Major loss of function.
          • Open

          Activity

            People

              rohini Rohini Palaniswamy
              rohini Rohini Palaniswamy
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: