Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4558

TestJobTrackerSafeMode is failing

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 1.1.0
        • Fix Version/s: 1.1.0
        • Component/s: None
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          MAPREDUCE-1906 exposed an issue with this unit test. It has 3 TTs running, but has a check for the TT count to reach exactly 2 (which would be reached with a higher heartbeat interval).

          The test ends up getting stuck, with the following message repeated multiple times.

              [junit] 2012-08-15 11:26:46,299 INFO  mapred.TestJobTrackerSafeMode (TestJobTrackerSafeMode.java:checkTrackers(201)) - Waiting for Initialize all Task Trackers
    [junit] 2012-08-15 11:26:47,301 INFO  mapred.TestJobTrackerSafeMode (TestJobTrackerSafeMode.java:checkTrackers(201)) - Waiting for Initialize all Task Trackers
    [junit] 2012-08-15 11:26:48,302 INFO  mapred.TestJobTrackerSafeMode (TestJobTrackerSafeMode.java:checkTrackers(201)) - Waiting for Initialize all Task Trackers
    [junit] 2012-08-15 11:26:49,303 INFO  mapred.TestJobTrackerSafeMode (TestJobTrackerSafeMode.java:checkTrackers(201)) - Waiting for Initialize all Task Trackers
          1. Text File
            MR4558.txt
            1.0 kB
            Siddharth Seth

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            sseth Siddharth Seth added a comment -

            Trivial patch - which changes the check to 3 TTs instead of 2. (Also disables the test)

            The test passes, but ends up generating cyclic attempt dir symlinks, which cause subsequent ant operations to fail.

            build/test/logs/userlogs/job_20120815112512392_0001/attempt_20120815112512392_0001_m_000000_0/attempt_20120815112512392_0001_m_000000_0/attempt_20120815112512392_0001_m_000000_0/..

            The unit test was enabled as part of MAPREDUCE-3837. I'd like to disable it again for now, with a separate jira to fix the symlinks.

            Show
            sseth Siddharth Seth added a comment - Trivial patch - which changes the check to 3 TTs instead of 2. (Also disables the test) The test passes, but ends up generating cyclic attempt dir symlinks, which cause subsequent ant operations to fail. build/test/logs/userlogs/job_20120815112512392_0001/attempt_20120815112512392_0001_m_000000_0/attempt_20120815112512392_0001_m_000000_0/attempt_20120815112512392_0001_m_000000_0/.. The unit test was enabled as part of MAPREDUCE-3837 . I'd like to disable it again for now, with a separate jira to fix the symlinks.
            Hide
            Permalink
            acmurthy Arun C Murthy added a comment -

            The unit test was enabled as part of MAPREDUCE-3837. I'd like to disable it again for now, with a separate jira to fix the symlinks.

            +1, this test seems like it needs more work. Mayank, will you have time to look at this? Thanks.

            Show
            acmurthy Arun C Murthy added a comment - The unit test was enabled as part of MAPREDUCE-3837 . I'd like to disable it again for now, with a separate jira to fix the symlinks. +1, this test seems like it needs more work. Mayank, will you have time to look at this? Thanks.
            Hide
            Permalink
            acmurthy Arun C Murthy added a comment -

            Pinging Mayank after adding him to the watch list... that should help!

            Show
            acmurthy Arun C Murthy added a comment - Pinging Mayank after adding him to the watch list... that should help!
            Hide
            Permalink
            mayank_bansal Mayank Bansal added a comment -

            yeah sure, I can take a look.

            Sid,

            Did you create a JIRA or I can do that as well?

            Thanks,
            Mayank

            Show
            mayank_bansal Mayank Bansal added a comment - yeah sure, I can take a look. Sid, Did you create a JIRA or I can do that as well? Thanks, Mayank
            Hide
            Permalink
            sseth Siddharth Seth added a comment -

            Committed to branch-1. Thanks for the review Arun.

            Mayank, I've created MAPREDUCE-4563 for the symlinks..

            Show
            sseth Siddharth Seth added a comment - Committed to branch-1. Thanks for the review Arun. Mayank, I've created MAPREDUCE-4563 for the symlinks..
            Hide
            Permalink
            acmurthy Arun C Murthy added a comment -

            Matt - if you don't mind, I'd like to merge this into branch-1.1 since it's been well baked-in. Thoughts?

            Show
            acmurthy Arun C Murthy added a comment - Matt - if you don't mind, I'd like to merge this into branch-1.1 since it's been well baked-in. Thoughts?
            Hide
            Permalink
            acmurthy Arun C Murthy added a comment -

            I merged this to branch-1.1 too.

            Show
            acmurthy Arun C Murthy added a comment - I merged this to branch-1.1 too.
            Hide
            Permalink
            mattf Matt Foley added a comment -

            Accepted.

            Show
            mattf Matt Foley added a comment - Accepted.
            Hide
            Permalink
            mattf Matt Foley added a comment -

            Closed upon release of Hadoop-1.1.0.

            Show
            mattf Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

              People

              • Assignee:
                sseth Siddharth Seth
                Reporter:
                sseth Siddharth Seth
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development