Hadoop Common
  1. Hadoop Common
  2. HADOOP-9217

Print thread dumps when hadoop-common tests fail

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha, 0.23.5
    • Fix Version/s: 2.0.3-alpha, 0.23.6
    • Component/s: test
    • Labels:
      None

      Description

      Printing thread dumps when tests fail due to timeouts was introduced in HADOOP-8755, but was enabled in M/R, HDFS and Yarn only.
      It makes sense to enable in hadoop-common as well. In particular, TestZKFailoverController seems to be one of the most flaky tests in trunk currently and having thread dumps may help debugging this.

      1. HADOOP-9217-fix1.patch
        1 kB
        Andrey Klochkov
      2. HADOOP-9217.patch
        0.7 kB
        Andrey Klochkov

        Issue Links

          Activity

          Hide
          Andrey Klochkov added a comment -

          The patch can be applied to all 3 branches (trunk, branch-2 and branch-0.23)

          Show
          Andrey Klochkov added a comment - The patch can be applied to all 3 branches (trunk, branch-2 and branch-0.23)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565014/HADOOP-9217.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The applied patch generated 2022 javac compiler warnings (more than the trunk's current 2014 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565014/HADOOP-9217.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The applied patch generated 2022 javac compiler warnings (more than the trunk's current 2014 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2050//console This message is automatically generated.
          Hide
          Andrey Klochkov added a comment -

          No tests are necessary as this is a change in build scripts.

          Show
          Andrey Klochkov added a comment - No tests are necessary as this is a change in build scripts.
          Hide
          Suresh Srinivas added a comment -

          +1 for the change.

          Show
          Suresh Srinivas added a comment - +1 for the change.
          Hide
          Suresh Srinivas added a comment -

          Committed the patch to trunk, branch-2 and 0.23.

          Thank you Andrey.

          Show
          Suresh Srinivas added a comment - Committed the patch to trunk, branch-2 and 0.23. Thank you Andrey.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3245 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3245/)
          HADOOP-9217. Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3245 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3245/ ) HADOOP-9217 . Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #98 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/98/)
          HADOOP-9217. Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #98 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/98/ ) HADOOP-9217 . Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #496 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/496/)
          HADOOP-9217. Merging change 1433713 from trunk (Revision 1433718)

          Result = FAILURE
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433718
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #496 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/496/ ) HADOOP-9217 . Merging change 1433713 from trunk (Revision 1433718) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433718 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1287 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1287/)
          HADOOP-9217. Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713)

          Result = FAILURE
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1287 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1287/ ) HADOOP-9217 . Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1315 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1315/)
          HADOOP-9217. Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713)

          Result = FAILURE
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1315 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1315/ ) HADOOP-9217 . Print thread dumps when hadoop-common tests fail. Contributed by Andrey Klochkov. (Revision 1433713) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433713 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          Hide
          Andrey Klochkov added a comment -

          Unfortunately with this patch I introduced a duplicated configuration of Surefire plugin in hadoop-common/pom.xml, effectively discarding a part of configuration. Attaching a patch which fixes this.

          Show
          Andrey Klochkov added a comment - Unfortunately with this patch I introduced a duplicated configuration of Surefire plugin in hadoop-common/pom.xml, effectively discarding a part of configuration. Attaching a patch which fixes this.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12566354/HADOOP-9217-fix1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2085//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2085//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12566354/HADOOP-9217-fix1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2085//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2085//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Andrey, thanks a lot for noticing this and providing a fix. Since this is a newly-discovered bug, I think we should open a new JIRA for this change.

          Show
          Aaron T. Myers added a comment - Hey Andrey, thanks a lot for noticing this and providing a fix. Since this is a newly-discovered bug, I think we should open a new JIRA for this change.
          Hide
          Andrey Klochkov added a comment -

          Good point, I'll open a separate one. Thanks.

          Show
          Andrey Klochkov added a comment - Good point, I'll open a separate one. Thanks.
          Hide
          Andrey Klochkov added a comment -

          HADOOP-9242 is created to track the additional fix

          Show
          Andrey Klochkov added a comment - HADOOP-9242 is created to track the additional fix

            People

            • Assignee:
              Andrey Klochkov
              Reporter:
              Andrey Klochkov
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development