Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1354

Incremental enhancements to the JobTracker for better scalability

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.

      Description

      It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.

      1. MAPREDUCE-1354_yhadoop20.patch
        5 kB
        Arun C Murthy
      2. MAPREDUCE-1354_yhadoop20.patch
        15 kB
        Arun C Murthy
      3. MAPREDUCE-1354_yhadoop20.patch
        16 kB
        Arun C Murthy
      4. MAPREDUCE-1354_yhadoop20.patch
        19 kB
        Arun C Murthy
      5. MAPREDUCE-1354_yhadoop20.patch
        21 kB
        Arun C Murthy
      6. MAPREDUCE-1354_yhadoop20.patch
        23 kB
        Arun C Murthy
      7. MAPREDUCE-1354_yhadoop20.patch
        24 kB
        Arun C Murthy
      8. mr-1354-y20.patch
        23 kB
        Hemanth Yamijala
      9. mapreduce-1354--2010-03-10.patch
        26 kB
        Dick King
      10. mapreduce-1354--2010-05-13.patch
        26 kB
        Dick King

        Activity

          People

          • Assignee:
            Dick King
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development