Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-380

The test TestMiniMRWithDFS.checkTaskDirectories is checking for task directories incorrectly

    Details

    • Type: Test Test
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The check for task directories in TestMiniMRWithDFS.checkTaskDirectories seems incorrect. The following code:

      assertTrue("task tracker dir " + trackerDir + " does not exist.", 
                       trackerDir.isDirectory());
      

      in the test case seems to assume that every configured task tracker has the task tracker directory TaskTracker.SUBDIR created. However, this directory is only created if any task is assigned to the task tracker. As there is no real control over which task trackers will actually be assigned tasks, it seems an incorrect check.

        Activity

        Hide
        Hemanth Yamijala added a comment -

        The issue is somewhat random, though it is replicable on some machines consistently. One way I could consistently replicate it is to modify the TestMiniMRWithDFS.testWithDFS method to run only the word count example, and comment out the PI example. Such a modification caused the following failure:

        task tracker dir /home/yhemanth/work/hadoop/hadoop-trunk/build/test/mapred/local/1_0/taskTracker does not exist.
        junit.framework.AssertionFailedError: task tracker dir /home/yhemanth/work/hadoop/hadoop-trunk/build/test/mapred/local/1_0/taskTracker does not exist.
            at org.apache.hadoop.mapred.TestMiniMRWithDFS.checkTaskDirectories(TestMiniMRWithDFS.java:140)
            at org.apache.hadoop.mapred.TestMiniMRWithDFS.runWordCount(TestMiniMRWithDFS.java:196)
            at org.apache.hadoop.mapred.TestMiniMRWithDFS.testWithDFS(TestMiniMRWithDFS.java:244)
        
        Show
        Hemanth Yamijala added a comment - The issue is somewhat random, though it is replicable on some machines consistently. One way I could consistently replicate it is to modify the TestMiniMRWithDFS.testWithDFS method to run only the word count example, and comment out the PI example. Such a modification caused the following failure: task tracker dir /home/yhemanth/work/hadoop/hadoop-trunk/build/test/mapred/local/1_0/taskTracker does not exist. junit.framework.AssertionFailedError: task tracker dir /home/yhemanth/work/hadoop/hadoop-trunk/build/test/mapred/local/1_0/taskTracker does not exist. at org.apache.hadoop.mapred.TestMiniMRWithDFS.checkTaskDirectories(TestMiniMRWithDFS.java:140) at org.apache.hadoop.mapred.TestMiniMRWithDFS.runWordCount(TestMiniMRWithDFS.java:196) at org.apache.hadoop.mapred.TestMiniMRWithDFS.testWithDFS(TestMiniMRWithDFS.java:244)

          People

          • Assignee:
            Unassigned
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development