Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-502

Allow jobtracker to be configured with zero completed jobs in memory

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      If the number of jobs per user exceeded mapred.jobtracker.completeuserjobs.maximum then the job was flushed out of the jobtracker's memory after the job finishes min-time (hardcoded to 1 min). This caused jobclient's fail with NPE. In this patch the min-time to retain a job is made configurable (mapred.jobtracker.retirejob.interval.min).
      Show
      If the number of jobs per user exceeded mapred.jobtracker.completeuserjobs.maximum then the job was flushed out of the jobtracker's memory after the job finishes min-time (hardcoded to 1 min). This caused jobclient's fail with NPE. In this patch the min-time to retain a job is made configurable (mapred.jobtracker.retirejob.interval.min).

      Description

      There is no way to specify that the jobtracker should not keep any completed job in memory.

      1. HADOOP-4973-v2.1.patch
        10 kB
        Amar Kamat
      2. HADOOP-4973-v2.2.patch
        11 kB
        Amar Kamat
      3. MAPREDUCE-502-v1.0.patch
        11 kB
        Amar Kamat
      4. MAPREDUCE-502-v1.1.patch
        11 kB
        Amar Kamat
      5. MAPREDUCE-502-v1.1-branch-0.20-example.2.patch
        7 kB
        Amar Kamat

        Activity

        Amar Kamat created issue -
        Amar Kamat made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]
        Amar Kamat made changes -
        Resolution Won't Fix [ 2 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Assignee Amar Kamat [ amar_kamat ]
        Amar Kamat made changes -
        Attachment HADOOP-4973-v2.1.patch [ 12411217 ]
        Amar Kamat made changes -
        Attachment HADOOP-4973-v2.2.patch [ 12411223 ]
        Owen O'Malley made changes -
        Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
        Key HADOOP-4973 MAPREDUCE-502
        Affects Version/s 0.21.0 [ 12313563 ]
        Component/s mapred [ 12310690 ]
        Fix Version/s 0.21.0 [ 12313563 ]
        Amar Kamat made changes -
        Attachment MAPREDUCE-502-v1.0.patch [ 12411376 ]
        Amar Kamat made changes -
        Attachment MAPREDUCE-502-v1.1.patch [ 12411529 ]
        Sharad Agarwal made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Fix Version/s 0.21.0 [ 12314045 ]
        Resolution Fixed [ 1 ]
        Amar Kamat made changes -
        Amar Kamat made changes -
        Release Note If the number of jobs per user exceeded mapred.jobtracker.completeuserjobs.maximum then the job was flushed out of the jobtracker's memory after the job finishes min-time (hardcoded to 1 min). This caused jobclient's fail with NPE. In this patch the min-time to retain a job is made configurable (mapred.jobtracker.retirejob.interval.min).
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Amar Kamat
            Reporter:
            Amar Kamat
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development