Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1354

Incremental enhancements to the JobTracker for better scalability

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.

      Description

      It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.

        Attachments

        1. MAPREDUCE-1354_yhadoop20.patch
          24 kB
          Arun C Murthy
        2. MAPREDUCE-1354_yhadoop20.patch
          23 kB
          Arun C Murthy
        3. MAPREDUCE-1354_yhadoop20.patch
          21 kB
          Arun C Murthy
        4. MAPREDUCE-1354_yhadoop20.patch
          19 kB
          Arun C Murthy
        5. MAPREDUCE-1354_yhadoop20.patch
          16 kB
          Arun C Murthy
        6. MAPREDUCE-1354_yhadoop20.patch
          15 kB
          Arun C Murthy
        7. MAPREDUCE-1354_yhadoop20.patch
          5 kB
          Arun C Murthy
        8. mapreduce-1354--2010-03-10.patch
          26 kB
          Dick King
        9. mapreduce-1354--2010-05-13.patch
          26 kB
          Dick King
        10. mr-1354-y20.patch
          23 kB
          Hemanth Yamijala

          Activity

            People

            • Assignee:
              dking Dick King
              Reporter:
              devaraj Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: