Hadoop Common
  1. Hadoop Common
  2. HADOOP-4372

Improve the way the job history files are managed during job recovery

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Today we use the .recover technique to handle the job history files when the jobtracker restarts. The comment here proposes a better way to handle the files.

      1. HADOOP-4372-v1.patch
        15 kB
        Amar Kamat
      2. HADOOP-4372-v1.4.patch
        17 kB
        Amar Kamat
      3. HADOOP-4372-v3.0.patch
        3 kB
        Amar Kamat
      4. HADOOP-4372-v3.1.patch
        7 kB
        Amar Kamat

        Activity

        Hide
        Amar Kamat added a comment -

        Attaching a patch the implements the basic idea. This patch uses a new filename on recovery. Updated the testcase accordingly.

        Show
        Amar Kamat added a comment - Attaching a patch the implements the basic idea. This patch uses a new filename on recovery. Updated the testcase accordingly.
        Hide
        Amar Kamat added a comment -

        Result of test-patch

        [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     +1 tests included.  The patch appears to include 6 new or modified tests.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        Ant tests passed on my box.

        Show
        Amar Kamat added a comment - Result of test-patch [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 6 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Ant tests passed on my box.
        Hide
        Devaraj Das added a comment -

        Could we instead pass a boolean to logSubmitted depending on the restart-count value (0 == new job, and the boolean is true), and then create the recovery file inside the jobHistory if the boolean is false.
        The problem with the current patch is that the filename of the history file changes upon restart..

        Show
        Devaraj Das added a comment - Could we instead pass a boolean to logSubmitted depending on the restart-count value (0 == new job, and the boolean is true), and then create the recovery file inside the jobHistory if the boolean is false. The problem with the current patch is that the filename of the history file changes upon restart..
        Hide
        Amar Kamat added a comment -

        Attaching a patch that simply uses the boolean passed whether the job is old or new. Testing in progress.

        Show
        Amar Kamat added a comment - Attaching a patch that simply uses the boolean passed whether the job is old or new. Testing in progress.
        Hide
        Amar Kamat added a comment -

        Changes to src/mapred/mapred-default.conf are not required. Plz ignore that.

        Show
        Amar Kamat added a comment - Changes to src/mapred/mapred-default.conf are not required. Plz ignore that.
        Hide
        Amar Kamat added a comment -

        Attaching a patch that optimizes jobhistory for new jobs. Result of test-patch

        [exec] -1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no tests are needed for this patch.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
        

        Ant test passed on my box

        Show
        Amar Kamat added a comment - Attaching a patch that optimizes jobhistory for new jobs. Result of test-patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Ant test passed on my box
        Hide
        Ravi Gummadi added a comment -

        Patch looks good.
        +1

        Show
        Ravi Gummadi added a comment - Patch looks good. +1
        Hide
        Devaraj Das added a comment -

        I just committed this. Thanks, Amar!

        Show
        Devaraj Das added a comment - I just committed this. Thanks, Amar!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk #830 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/830/)
        . Improves the way history filenames are obtained and manipulated. Contributed by Amar Kamat.

        Show
        Hudson added a comment - Integrated in Hadoop-trunk #830 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/830/ ) . Improves the way history filenames are obtained and manipulated. Contributed by Amar Kamat.
        Hide
        Devaraj Das added a comment -

        I committed this to 0.20 branch as well. There have been problems like job submission taking a long time when the number of files in the history folder are too large. This patch introduced an API that would fasten the process of obtaining a history file for a new job (earlier it used to do a scan of the history folder which is costly when the number of files are large).

        Show
        Devaraj Das added a comment - I committed this to 0.20 branch as well. There have been problems like job submission taking a long time when the number of files in the history folder are too large. This patch introduced an API that would fasten the process of obtaining a history file for a new job (earlier it used to do a scan of the history folder which is costly when the number of files are large).
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #863 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/863/ )

          People

          • Assignee:
            Amar Kamat
            Reporter:
            Amar Kamat
          • Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development