Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3852

test TestLinuxResourceCalculatorPlugin failing

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.24.0, 0.23.2
    • Fix Version/s: 0.23.2
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      tests are failing:
      org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin.testParsingProcStatAndCpuFile
      org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin.testParsingProcMemFile

      https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1831/testReport/junit/org.apache.hadoop.yarn.util/TestLinuxResourceCalculatorPlugin/testParsingProcStatAndCpuFile/

      both with similar error:
      java.io.FileNotFoundException: /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/MEMINFO_238849741 (No such file or directory)

      1. MAPREDUCE-3852.patch
        0.9 kB
        Thomas Graves

        Issue Links

          Activity

          Hide
          Thomas Graves added a comment -

          I think this was introduced by https://issues.apache.org/jira/browse/HADOOP-8035 when it removed the hadoop-project/pom.xml plugin to create-testdirs.

          Show
          Thomas Graves added a comment - I think this was introduced by https://issues.apache.org/jira/browse/HADOOP-8035 when it removed the hadoop-project/pom.xml plugin to create-testdirs.
          Hide
          Thomas Graves added a comment -

          here is patch that reverts the change and fixes the tests. it doesn't seem to affect build times much. I didn't look into other ways to solve but if someone wants to take a look, go for it.

          build command: mvn clean install site package -Pdist -DskipTests -Dmaven.javadoc.skip=true

          ran multiple times, with this change it seems to take 4 to 10 seconds longer then without it. That is out of about 7 minutes build time.

          Show
          Thomas Graves added a comment - here is patch that reverts the change and fixes the tests. it doesn't seem to affect build times much. I didn't look into other ways to solve but if someone wants to take a look, go for it. build command: mvn clean install site package -Pdist -DskipTests -Dmaven.javadoc.skip=true ran multiple times, with this change it seems to take 4 to 10 seconds longer then without it. That is out of about 7 minutes build time.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12514193/MAPREDUCE-3852.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 javac. The patch appears to cause tar ant target to fail.

          -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse.

          -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed the unit tests build

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1843//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1843//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12514193/MAPREDUCE-3852.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1843//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1843//console This message is automatically generated.
          Hide
          Thomas Graves added a comment -

          The jenkins build patched the wrong file. This patch is for the top level hadoop-project/pom.xml. The build does a patch -p1 which patched hadoop-mapreduce-project/pom.xml

          Show
          Thomas Graves added a comment - The jenkins build patched the wrong file. This patch is for the top level hadoop-project/pom.xml. The build does a patch -p1 which patched hadoop-mapreduce-project/pom.xml
          Hide
          Mahadev konar added a comment -

          Thanks Thomas. Committing now!

          Show
          Mahadev konar added a comment - Thanks Thomas. Committing now!
          Hide
          Mahadev konar added a comment -

          I just committed this. Thanks Thomas. Am kicking off the trunk and 0.23 builds. Hopefully they are back to normal.

          Show
          Mahadev konar added a comment - I just committed this. Thanks Thomas. Am kicking off the trunk and 0.23 builds. Hopefully they are back to normal.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1719 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1719/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418)

          Result = SUCCESS
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1719 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1719/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418) Result = SUCCESS mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1793 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1793/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418)

          Result = SUCCESS
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1793 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1793/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418) Result = SUCCESS mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #543 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/543/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420)

          Result = SUCCESS
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #543 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/543/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420) Result = SUCCESS mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #531 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/531/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420)

          Result = SUCCESS
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #531 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/531/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420) Result = SUCCESS mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1730 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1730/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418)

          Result = ABORTED
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1730 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1730/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418) Result = ABORTED mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #987 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/987/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418)

          Result = FAILURE
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #987 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/987/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418) Result = FAILURE mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #190 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/190/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420)

          Result = FAILURE
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #190 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/190/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420) Result = FAILURE mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #547 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/547/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420)

          Result = ABORTED
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #547 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/547/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420) Result = ABORTED mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #954 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/954/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418)

          Result = FAILURE
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #954 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/954/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) (Revision 1243418) Result = FAILURE mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243418 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #167 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/167/)
          MAPREDUCE-3852. Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420)

          Result = FAILURE
          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #167 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/167/ ) MAPREDUCE-3852 . Test TestLinuxResourceCalculatorPlugin failing. (Thomas Graves via mahadev) - Merging r1243418 from trunk. (Revision 1243420) Result = FAILURE mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1243420 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml

            People

            • Assignee:
              Thomas Graves
              Reporter:
              Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development