Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1053

MRReliabilityTest does not kill/fail tasks if history is enabled

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.20.1
    • Component/s: jobtracker
    • Labels:
      None

      Description

      When history is enabled, MRReliabilityTest fails to fail/kill tasks. Also the scenario of lost TTs is not being tested.

        Activity

        Ramya Sunil created issue -
        Hide
        Ramya Sunil added a comment -

        Below is the output of MRReliability, when history is enabled:

        mapred.ReliabilityTest: Waiting for the job org.apache.hadoop.examples.SleepJob to start
        mapred.JobClient: Running job:<jobID>
        mapred.ReliabilityTest: Will kill tasks based on Maps' progress
        mapred.ReliabilityTest: Initial progress threshold: 0.2. Threshold Multiplier: 2. Number of iterations: 2
        mapred.ReliabilityTest: Will kill tasks based on Reduces' progress
        mapred.ReliabilityTest: Initial progress threshold: 0.2. Threshold Multiplier: 2. Number of iterations: 2
        mapred.ReliabilityTest: DONE WITH THE TASK KILL/FAIL TESTS
        mapred.ReliabilityTest: Will STOP/RESUME tasktrackers based on Maps' progress
        mapred.ReliabilityTest: Initial progress threshold: 0.4. Threshold Multiplier: 2. Number of iterations: 1
        mapred.ReliabilityTest: Will STOP/RESUME tasktrackers based on Reduces' progress
        mapred.ReliabilityTest: Initial progress threshold: 0.4. Threshold Multiplier: 2. Number of iterations: 1
        mapred.ReliabilityTest: DONE WITH THE TESTS TO DO WITH LOST TASKTRACKERS
        mapred.JobClient: map 0% reduce 0%
        mapred.JobClient: map 1% reduce 0%
        mapred.JobClient: map 2% reduce 0%

        In the above output, statements such as "DONE WITH THE TASK KILL/FAIL TESTS" and "DONE WITH THE TESTS TO DO WITH LOST TASKTRACKERS" are logged even before the test starts. And also, all through the test there were no task failures or lost TTs observed. Hence the MRReliabilityTest is broken when history is enabled.

        Show
        Ramya Sunil added a comment - Below is the output of MRReliability, when history is enabled: mapred.ReliabilityTest: Waiting for the job org.apache.hadoop.examples.SleepJob to start mapred.JobClient: Running job:<jobID> mapred.ReliabilityTest: Will kill tasks based on Maps' progress mapred.ReliabilityTest: Initial progress threshold: 0.2. Threshold Multiplier: 2. Number of iterations: 2 mapred.ReliabilityTest: Will kill tasks based on Reduces' progress mapred.ReliabilityTest: Initial progress threshold: 0.2. Threshold Multiplier: 2. Number of iterations: 2 mapred.ReliabilityTest: DONE WITH THE TASK KILL/FAIL TESTS mapred.ReliabilityTest: Will STOP/RESUME tasktrackers based on Maps' progress mapred.ReliabilityTest: Initial progress threshold: 0.4. Threshold Multiplier: 2. Number of iterations: 1 mapred.ReliabilityTest: Will STOP/RESUME tasktrackers based on Reduces' progress mapred.ReliabilityTest: Initial progress threshold: 0.4. Threshold Multiplier: 2. Number of iterations: 1 mapred.ReliabilityTest: DONE WITH THE TESTS TO DO WITH LOST TASKTRACKERS mapred.JobClient: map 0% reduce 0% mapred.JobClient: map 1% reduce 0% mapred.JobClient: map 2% reduce 0% In the above output, statements such as "DONE WITH THE TASK KILL/FAIL TESTS" and "DONE WITH THE TESTS TO DO WITH LOST TASKTRACKERS" are logged even before the test starts. And also, all through the test there were no task failures or lost TTs observed. Hence the MRReliabilityTest is broken when history is enabled.
        Hide
        Hemanth Yamijala added a comment -

        Marking as duplicate of MAPREDUCE-1062.

        Show
        Hemanth Yamijala added a comment - Marking as duplicate of MAPREDUCE-1062 .
        Hemanth Yamijala made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Duplicate [ 3 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        2d 2h 52m 1 Hemanth Yamijala 06/Oct/09 16:53

          People

          • Assignee:
            Unassigned
            Reporter:
            Ramya Sunil
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development