Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1354

Incremental enhancements to the JobTracker for better scalability

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.22.0
    • jobtracker
    • None
    • Reviewed
    • Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.

    Description

      It'd be nice to have the JobTracker object not be locked while accessing the HDFS for reading the jobconf file and while writing the jobinfo file in the submitJob method. We should see if we can avoid taking the lock altogether.

      Attachments

        1. MAPREDUCE-1354_yhadoop20.patch
          5 kB
          Arun Murthy
        2. MAPREDUCE-1354_yhadoop20.patch
          15 kB
          Arun Murthy
        3. MAPREDUCE-1354_yhadoop20.patch
          16 kB
          Arun Murthy
        4. MAPREDUCE-1354_yhadoop20.patch
          19 kB
          Arun Murthy
        5. MAPREDUCE-1354_yhadoop20.patch
          21 kB
          Arun Murthy
        6. MAPREDUCE-1354_yhadoop20.patch
          23 kB
          Arun Murthy
        7. MAPREDUCE-1354_yhadoop20.patch
          24 kB
          Arun Murthy
        8. mr-1354-y20.patch
          23 kB
          Hemanth Yamijala
        9. mapreduce-1354--2010-03-10.patch
          26 kB
          Dick King
        10. mapreduce-1354--2010-05-13.patch
          26 kB
          Dick King

        Activity

          People

            dking Dick King
            ddas Devaraj Das
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: