Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 0.23.1
    • mr-am, mrv2, performance
    • None
    • Reviewed

    Description

      MR AM creates new JobConf objects unnecessarily in a couple of places in JobImpl and TaskImpl which occupy non-trivial amount of heap.

      While working with a 64 bit JVM on 100K maps jobs, with uncompressed pointers, removing those extraneous objects helped in addressing OOM with 2GB AM heap size.

      Attachments

        1. MAPREDUCE-3567-20111215.1.txt
          25 kB
          Vinod Kumar Vavilapalli
        2. MAPREDUCE-3567-20111222.txt
          26 kB
          Vinod Kumar Vavilapalli

        Activity

          People

            vinodkv Vinod Kumar Vavilapalli
            vinodkv Vinod Kumar Vavilapalli
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: