XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • tez
    • None
    • Reviewed

    Description

      We use map jvm opts for root vertices and reduce jvm opts for intermediate and leaf vertices. For a simple order by or skewed join the Sampler will be a root vertex and Partitioner will be a intermediate vertex. If the map opts and reduce opts are different containers will not be reused and there will be lot of resource wastage when the reduce opts specify higher memory than map which is usually the case. The partitioner does not do much and just does identity mapping. So we should be good using map opts there.

      Attachments

        1. PIG-4245-1.patch
          10 kB
          Rohini Palaniswamy

        Activity

          People

            rohini Rohini Palaniswamy
            rohini Rohini Palaniswamy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: