Hadoop Common
  1. Hadoop Common
  2. HADOOP-8755

Print thread dump when tests fail due to timeout

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 0.23.1, 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.5
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When a test fails due to timeout it's often not clear what is the root cause. See HDFS-3364 as an example.

      We can print dump of all threads in this case, this may help finding causes.

      1. HDFS-3762.patch
        3 kB
        Andrey Klochkov
      2. HDFS-3762.patch
        3 kB
        Andrey Klochkov
      3. HDFS-3762.patch
        2 kB
        Andrey Klochkov
      4. HDFS-3762.patch
        3 kB
        Ilya Katsov
      5. HDFS-3762-branch-0.23.patch
        6 kB
        Andrey Klochkov
      6. HDFS-3762.patch
        6 kB
        Andrey Klochkov
      7. HADOOP-8755.patch
        14 kB
        Andrey Klochkov
      8. HADOOP-8755.patch
        14 kB
        Andrey Klochkov
      9. HADOOP-8755.patch
        15 kB
        Aaron T. Myers

        Issue Links

          Activity

          Hide
          Andrey Klochkov added a comment -

          Attaching a patch. It's based on junit's run listeners to print thread dumps. I couldn't make it work until switched from surefire 2.10 to 2.12.

          Show
          Andrey Klochkov added a comment - Attaching a patch. It's based on junit's run listeners to print thread dumps. I couldn't make it work until switched from surefire 2.10 to 2.12.
          Hide
          Todd Lipcon added a comment -

          Can we do this project-wide? i.e in common and MR too? This is great. I've been looking for a way to do this for some time.

          Show
          Todd Lipcon added a comment - Can we do this project-wide? i.e in common and MR too? This is great. I've been looking for a way to do this for some time.
          Hide
          Andrey Klochkov added a comment -

          Sure it's possible to do it project wide. Attaching updated patch.

          Show
          Andrey Klochkov added a comment - Sure it's possible to do it project wide. Attaching updated patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539122/HDFS-3762.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2952//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539122/HDFS-3762.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2952//console This message is automatically generated.
          Hide
          Andrey Klochkov added a comment -

          Sorry, the previous patch was for 0.23. This one is for trunk.

          Show
          Andrey Klochkov added a comment - Sorry, the previous patch was for 0.23. This one is for trunk.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539139/HDFS-3762.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2955//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2955//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539139/HDFS-3762.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2955//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2955//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12543235/HDFS-3762.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3132//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3132//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12543235/HDFS-3762.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3132//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3132//console This message is automatically generated.
          Hide
          Andrey Klochkov added a comment -

          Found that hadoop-auth won't run "test" target due to the class TestTimedOutListener absence in the classpath. Turns out that there are projects which do not depend on hadoop-common. Changed the patch to enable the listener in specific projects (HDFS, M/R and Yarn) instead of doing this in hadoop-project/pom.xml

          Show
          Andrey Klochkov added a comment - Found that hadoop-auth won't run "test" target due to the class TestTimedOutListener absence in the classpath. Turns out that there are projects which do not depend on hadoop-common. Changed the patch to enable the listener in specific projects (HDFS, M/R and Yarn) instead of doing this in hadoop-project/pom.xml
          Hide
          Aaron T. Myers added a comment -

          I've moved this JIRA to Common instead of HDFS, since we're doing it for all the projects.

          I'll take a look at this shortly.

          Show
          Aaron T. Myers added a comment - I've moved this JIRA to Common instead of HDFS, since we're doing it for all the projects. I'll take a look at this shortly.
          Hide
          Aaron T. Myers added a comment -

          Hey Andrey, can you comment on what testing of this patch you've done? I tried the following, but it didn't work as I expected:

          1. Applying the patch you posted here.
          2. Lower the configured test timeout in hadoop-project/pom.xml from 900 seconds to 5 seconds.
          3. Add a "Thread.sleep(10 * 1000)" (10 seconds) to an HDFS test case.
          4. Running the modified HDFS test case.

          The test timed out as I intended it to, but I couldn't find the thread dump output anywhere - not in the test output file, and not on the console. Where should I be looking, or what am I missing?

          Thanks a lot.

          Show
          Aaron T. Myers added a comment - Hey Andrey, can you comment on what testing of this patch you've done? I tried the following, but it didn't work as I expected: Applying the patch you posted here. Lower the configured test timeout in hadoop-project/pom.xml from 900 seconds to 5 seconds. Add a "Thread.sleep(10 * 1000)" (10 seconds) to an HDFS test case. Running the modified HDFS test case. The test timed out as I intended it to, but I couldn't find the thread dump output anywhere - not in the test output file, and not on the console. Where should I be looking, or what am I missing? Thanks a lot.
          Hide
          Andrey Klochkov added a comment -

          Hi Aaron,
          You're right - changing this timeout wouldn't help. There are 2 timeouts for test execution time: one in surefire and another is in junit. Surefire just kills a child process when timeout is exceeded, and this patch doesn't handle this. What's handled is if a test method is annotated with @Test and the timeout parameter is given, then junit will fail the test and thread dump will be printed. TestFileAppend4 is an example of a test providing timeout parameter.

          When testing the patch I reduced timeout in TestFileAppend4 and validated that thread dump is presented in the test output file.

          AFAIK we can't really do anything with the surefire timeout. Still we may have thread dumps printed for all tests in case of timeouts if we introduce a default timeout for all tests on the junit level. I guess it is doable with a custom surefire provider for junit, but I'm not sure we really need this. What do you think?

          Show
          Andrey Klochkov added a comment - Hi Aaron, You're right - changing this timeout wouldn't help. There are 2 timeouts for test execution time: one in surefire and another is in junit. Surefire just kills a child process when timeout is exceeded, and this patch doesn't handle this. What's handled is if a test method is annotated with @Test and the timeout parameter is given, then junit will fail the test and thread dump will be printed. TestFileAppend4 is an example of a test providing timeout parameter. When testing the patch I reduced timeout in TestFileAppend4 and validated that thread dump is presented in the test output file. AFAIK we can't really do anything with the surefire timeout. Still we may have thread dumps printed for all tests in case of timeouts if we introduce a default timeout for all tests on the junit level. I guess it is doable with a custom surefire provider for junit, but I'm not sure we really need this. What do you think?
          Hide
          Aaron T. Myers added a comment -

          You're right - changing this timeout wouldn't help. There are 2 timeouts for test execution time: one in surefire and another is in junit. Surefire just kills a child process when timeout is exceeded, and this patch doesn't handle this.

          Got it. Makes sense. Thanks a lot for the explanation.

          What's handled is if a test method is annotated with @Test and the timeout parameter is given, then junit will fail the test and thread dump will be printed. TestFileAppend4 is an example of a test providing timeout parameter.

          Gotcha. I also tested this similarly and was able to confirm that the patch works as expected. Very cool.

          AFAIK we can't really do anything with the surefire timeout. Still we may have thread dumps printed for all tests in case of timeouts if we introduce a default timeout for all tests on the junit level. I guess it is doable with a custom surefire provider for junit, but I'm not sure we really need this. What do you think?

          I think that would be a great thing to add. Not very many of the Hadoop tests are annotated with a JUnit timeout (79 / 3984, by my quick count), and in my experience tests which aren't annotated with a JUnit timeout certainly do in fact timeout and end up hitting the Surefire fork timeout. If that's not handled somehow, that would make this change much less useful. If it's easy, mind adding a default JUnit timeout to the next rev of this patch? If not, we could certainly do it as a separate JIRA.

          As for comments on the contents of the patch, just a few little things:

          1. I think it'd be good to print an informative header before dumping the stacks, e.g. something like "====> TEST TIMED OUT. PRINTING THREAD DUMP. <===="
          2. Several of the lines in the patch go over 80 characters.
          3. Our coding guidelines say that lines which continue onto the next line should be indented 4 spaces, not 2.

          Otherwise the patch looks great.

          Show
          Aaron T. Myers added a comment - You're right - changing this timeout wouldn't help. There are 2 timeouts for test execution time: one in surefire and another is in junit. Surefire just kills a child process when timeout is exceeded, and this patch doesn't handle this. Got it. Makes sense. Thanks a lot for the explanation. What's handled is if a test method is annotated with @Test and the timeout parameter is given, then junit will fail the test and thread dump will be printed. TestFileAppend4 is an example of a test providing timeout parameter. Gotcha. I also tested this similarly and was able to confirm that the patch works as expected. Very cool. AFAIK we can't really do anything with the surefire timeout. Still we may have thread dumps printed for all tests in case of timeouts if we introduce a default timeout for all tests on the junit level. I guess it is doable with a custom surefire provider for junit, but I'm not sure we really need this. What do you think? I think that would be a great thing to add. Not very many of the Hadoop tests are annotated with a JUnit timeout (79 / 3984, by my quick count), and in my experience tests which aren't annotated with a JUnit timeout certainly do in fact timeout and end up hitting the Surefire fork timeout. If that's not handled somehow, that would make this change much less useful. If it's easy, mind adding a default JUnit timeout to the next rev of this patch? If not, we could certainly do it as a separate JIRA. As for comments on the contents of the patch, just a few little things: I think it'd be good to print an informative header before dumping the stacks, e.g. something like "====> TEST TIMED OUT. PRINTING THREAD DUMP. <====" Several of the lines in the patch go over 80 characters. Our coding guidelines say that lines which continue onto the next line should be indented 4 spaces, not 2. Otherwise the patch looks great.
          Hide
          Aaron T. Myers added a comment -

          Hi Andrey, do you think you'll have a chance to post another rev of this patch soon? I'm looking forward to getting this change checked in.

          Show
          Aaron T. Myers added a comment - Hi Andrey, do you think you'll have a chance to post another rev of this patch soon? I'm looking forward to getting this change checked in.
          Hide
          Andrey Klochkov added a comment -

          Hi Aaron, I'd like to make number of improvements before submitting a patch. These are:

          1. include dead lock detection into the dump
          2. introduce default timeouts on junit level

          The 2nd one is not easy. I'm thinking about 2 possible ways to implement it, and both seem pretty hackie. The first is implementing a custom Surefire provider. It's not straightforward (if possible) as there are no explicit extension points for that in Surefire. The second is doing instrumentation with a custom JVM agent, adding "timeout" parameter to the @Test annotation for all test methods which don't provide it. I'm planning to evaluate both ways but it may take time. I think a separate JIRA would be better for this part. WDYT?

          Show
          Andrey Klochkov added a comment - Hi Aaron, I'd like to make number of improvements before submitting a patch. These are: include dead lock detection into the dump introduce default timeouts on junit level The 2nd one is not easy. I'm thinking about 2 possible ways to implement it, and both seem pretty hackie. The first is implementing a custom Surefire provider. It's not straightforward (if possible) as there are no explicit extension points for that in Surefire. The second is doing instrumentation with a custom JVM agent, adding "timeout" parameter to the @Test annotation for all test methods which don't provide it. I'm planning to evaluate both ways but it may take time. I think a separate JIRA would be better for this part. WDYT?
          Hide
          Aaron T. Myers added a comment -

          Thanks for the update, Andrey. I agree that both of those improvements sound good, though both could reasonably be done in separate JIRAs.

          Regarding the difficulty of implementing option #2, I agree that both of those sound pretty hacky, probably to the degree that it's not worth it. I don't even think that dynamically modifying the @Test annotations for all the test methods would work, since I don't think you can change annotation attributes at run-time. I've also taken a look at the JUnit docs, and I think another way of setting a default timeout might be to implement a custom BlockJUnit4ClassRunner which overrides the withPotentialTimeout method to add a default value if none is set. That's still not trivial, but it seems a little less hacky than either of the two options so far proposed.

          All that said, given the difficulty of setting a default JUnit test timeout, I'd even be OK with just modifying all existing tests to set the timeout attribute of the @Test annotation, and going forward being sure to always set one. Considering we recently converted all of Hadoop's tests to JUnit 4 style, this seems like it might be reasonable. I think we could very close to such a patch just by doing the following:

          sed -i 's/@Test$/@Test(timeout=480000)/g' `egrep -r '@Test$' . -l`
          sed -i 's/@Test(expected/@Test(timeout=480000, expected/g' `egrep -r '@Test\(expected' . -l`
          

          Thoughts?

          Show
          Aaron T. Myers added a comment - Thanks for the update, Andrey. I agree that both of those improvements sound good, though both could reasonably be done in separate JIRAs. Regarding the difficulty of implementing option #2, I agree that both of those sound pretty hacky, probably to the degree that it's not worth it. I don't even think that dynamically modifying the @Test annotations for all the test methods would work, since I don't think you can change annotation attributes at run-time. I've also taken a look at the JUnit docs, and I think another way of setting a default timeout might be to implement a custom BlockJUnit4ClassRunner which overrides the withPotentialTimeout method to add a default value if none is set. That's still not trivial, but it seems a little less hacky than either of the two options so far proposed. All that said, given the difficulty of setting a default JUnit test timeout, I'd even be OK with just modifying all existing tests to set the timeout attribute of the @Test annotation, and going forward being sure to always set one. Considering we recently converted all of Hadoop's tests to JUnit 4 style, this seems like it might be reasonable. I think we could very close to such a patch just by doing the following: sed -i 's/@Test$/@Test(timeout=480000)/g' `egrep -r '@Test$' . -l` sed -i 's/@Test(expected/@Test(timeout=480000, expected/g' `egrep -r '@Test\(expected' . -l` Thoughts?
          Hide
          Andrey Klochkov added a comment -

          Aaron, I'll do the 1st improvement in this JIRA as it's really simple.

          Marking all the annotations with timeouts manually - agree, that's much simpler and more transparent. The downside is polluting all the tests with it. In general limiting true unit tests with timeouts isn't the right thing, and in our case we're just doing this to troubleshoot flakiness of particular component tests. How about not having timeouts by default and instead marking just those tests which fail intermittently?

          Show
          Andrey Klochkov added a comment - Aaron, I'll do the 1st improvement in this JIRA as it's really simple. Marking all the annotations with timeouts manually - agree, that's much simpler and more transparent. The downside is polluting all the tests with it. In general limiting true unit tests with timeouts isn't the right thing, and in our case we're just doing this to troubleshoot flakiness of particular component tests. How about not having timeouts by default and instead marking just those tests which fail intermittently?
          Hide
          Aaron T. Myers added a comment -

          That sounds fine to me, Andrey. Once this patch goes in, I'll send out a note to common-dev@ saying something along the lines of "We now have support for printing a thread dump whenever a test case times out. However, this will only happen for test cases which are annotated with a JUnit timeout. If you see a test case fail by reaching the Surefire fork timeout, please file a JIRA to add a JUnit timeout for that test. If when adding a test case you think it might time out, please add a JUnit timeout."

          Sound good?

          Show
          Aaron T. Myers added a comment - That sounds fine to me, Andrey. Once this patch goes in, I'll send out a note to common-dev@ saying something along the lines of "We now have support for printing a thread dump whenever a test case times out. However, this will only happen for test cases which are annotated with a JUnit timeout. If you see a test case fail by reaching the Surefire fork timeout, please file a JIRA to add a JUnit timeout for that test. If when adding a test case you think it might time out, please add a JUnit timeout." Sound good?
          Hide
          Andrey Klochkov added a comment -

          Sounds great! Thanks, Aaron. I'll update the patch shortly.

          Show
          Andrey Klochkov added a comment - Sounds great! Thanks, Aaron. I'll update the patch shortly.
          Hide
          Andrey Klochkov added a comment -

          Attaching an updated patch

          Show
          Andrey Klochkov added a comment - Attaching an updated patch
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12544912/HADOOP-8755.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          -1 javac. The applied patch generated 2067 javac compiler warnings (more than the trunk's current 2056 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.hdfs.TestPersistBlocks

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544912/HADOOP-8755.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. -1 javac. The applied patch generated 2067 javac compiler warnings (more than the trunk's current 2056 warnings). +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.hdfs.TestPersistBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1450//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Thanks for posting an updated patch, Andrey. The test failures are most likely unrelated, but the javac warning seems to be due to this patch. Mind taking a look into that?

          Show
          Aaron T. Myers added a comment - Thanks for posting an updated patch, Andrey. The test failures are most likely unrelated, but the javac warning seems to be due to this patch. Mind taking a look into that?
          Hide
          Andrey Klochkov added a comment -

          Fixed pom files to avoid warnings from Maven.

          Show
          Andrey Klochkov added a comment - Fixed pom files to avoid warnings from Maven.
          Hide
          Aaron T. Myers added a comment -

          Not quite sure why test-patch didn't run on this latest patch. Regardless, I've just kicked Jenkins manually.

          Show
          Aaron T. Myers added a comment - Not quite sure why test-patch didn't run on this latest patch. Regardless, I've just kicked Jenkins manually.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12544942/HADOOP-8755.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs:

          org.apache.hadoop.hdfs.TestDatanodeBlockScanner

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1454//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1454//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544942/HADOOP-8755.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs: org.apache.hadoop.hdfs.TestDatanodeBlockScanner +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1454//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1454//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Patch looks great, except that it needs to include the Apache License header in the new files. Here's an updated patch that adds those.

          +1, I'm going to commit this momentarily since the difference between this and the last is just comments.

          Show
          Aaron T. Myers added a comment - Patch looks great, except that it needs to include the Apache License header in the new files. Here's an updated patch that adds those. +1, I'm going to commit this momentarily since the difference between this and the last is just comments.
          Hide
          Aaron T. Myers added a comment -

          I've just committed this to trunk and branch-2.

          Thanks a lot for the contribution, Andrey!

          Show
          Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Andrey!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2731 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2731/)
          HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2731 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2731/ ) HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2794 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2794/)
          HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2794 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2794/ ) HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2755 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2755/)
          HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2755 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2755/ ) HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1165 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1165/)
          HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1165 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1165/ ) HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1196/)
          HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1196 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1196/ ) HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1384627) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1384627 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          Hide
          Aaron T. Myers added a comment -

          I've just sent the email to common-dev@ as discussed.

          Thanks again, Andrey.

          Show
          Aaron T. Myers added a comment - I've just sent the email to common-dev@ as discussed. Thanks again, Andrey.
          Hide
          Robert Joseph Evans added a comment -

          I just pulled this into branch-0.23

          Show
          Robert Joseph Evans added a comment - I just pulled this into branch-0.23
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #392 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/392/)
          svn merge -c 1384627 FIXES: HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1392458)

          Result = UNSTABLE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392458
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #392 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/392/ ) svn merge -c 1384627 FIXES: HADOOP-8755 . Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov. (Revision 1392458) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1392458 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TestTimedOutTestsListener.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/TimedOutTestsListener.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/pom.xml /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-yarn-project/hadoop-yarn/pom.xml

            People

            • Assignee:
              Andrey Klochkov
              Reporter:
              Andrey Klochkov
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development