Hadoop Common
  1. Hadoop Common
  2. HADOOP-8278

Make sure components declare correct set of dependencies

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.2-alpha
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As mentioned by Scott Carey in https://issues.apache.org/jira/browse/MAPREDUCE-3378?focusedCommentId=13173437&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13173437, we need to make sure that components are declaring the correct set of dependencies. In current trunk there are errors of omission and commission (as reported by 'mvn dependency:analyze'):

      • "Used undeclared dependencies" - these are dependencies that are being met transitively. They should be added explicitly as "compile" or "provided" scope.
      • "Unused declared dependencies" - these are dependencies that are not needed for compilation, although they may be needed at runtime. They certainly should not be "compile" scope - they should either be removed or marked as "runtime" or "test" scope.
      1. HADOOP-8278.patch
        21 kB
        Tom White
      2. HADOOP-8278.patch
        21 kB
        Tom White
      3. HADOOP-8278.patch
        21 kB
        Tom White
      4. HADOOP-8278.patch
        19 kB
        Tom White
      5. HADOOP-8278.patch
        17 kB
        Alejandro Abdelnur
      6. HADOOP-8278.patch
        17 kB
        Tom White

        Issue Links

          Activity

          Hide
          Eli Collins added a comment -

          I merged this to branch-2.1.0-alpha

          Show
          Eli Collins added a comment - I merged this to branch-2.1.0-alpha
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1168 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1168/)
          HADOOP-8278. Make sure components declare correct set of dependencies. (Revision 1373574)

          Result = FAILURE
          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          • /hadoop/common/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1168 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1168/ ) HADOOP-8278 . Make sure components declare correct set of dependencies. (Revision 1373574) Result = FAILURE tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml /hadoop/common/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1136 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1136/)
          HADOOP-8278. Make sure components declare correct set of dependencies. (Revision 1373574)

          Result = SUCCESS
          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          • /hadoop/common/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1136 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1136/ ) HADOOP-8278 . Make sure components declare correct set of dependencies. (Revision 1373574) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml /hadoop/common/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2609 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2609/)
          HADOOP-8278. Make sure components declare correct set of dependencies. (Revision 1373574)

          Result = FAILURE
          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          • /hadoop/common/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2609 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2609/ ) HADOOP-8278 . Make sure components declare correct set of dependencies. (Revision 1373574) Result = FAILURE tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml /hadoop/common/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2581 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2581/)
          HADOOP-8278. Make sure components declare correct set of dependencies. (Revision 1373574)

          Result = SUCCESS
          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          • /hadoop/common/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2581 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2581/ ) HADOOP-8278 . Make sure components declare correct set of dependencies. (Revision 1373574) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml /hadoop/common/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2646 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2646/)
          HADOOP-8278. Make sure components declare correct set of dependencies. (Revision 1373574)

          Result = SUCCESS
          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml
          • /hadoop/common/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2646 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2646/ ) HADOOP-8278 . Make sure components declare correct set of dependencies. (Revision 1373574) Result = SUCCESS tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373574 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth-examples/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestAtomicFileOutputStream.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/pom.xml /hadoop/common/trunk/pom.xml
          Hide
          Tom White added a comment -

          I've just committed this.

          Show
          Tom White added a comment - I've just committed this.
          Hide
          Tom White added a comment -

          I ran all the failed tests locally with the patch and they all passed. They seem to fail occasionally on jenkins.

          Show
          Tom White added a comment - I ran all the failed tests locally with the patch and they all passed. They seem to fail occasionally on jenkins.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12540724/HADOOP-8278.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples:

          org.apache.hadoop.ha.TestZKFailoverController
          org.apache.hadoop.hdfs.server.datanode.TestHSync
          org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
          org.apache.hadoop.hdfs.web.TestWebHDFS
          org.apache.hadoop.hdfs.server.datanode.TestBPOfferService
          org.apache.hadoop.hdfs.TestPersistBlocks

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1305//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1305//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540724/HADOOP-8278.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples: org.apache.hadoop.ha.TestZKFailoverController org.apache.hadoop.hdfs.server.datanode.TestHSync org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics org.apache.hadoop.hdfs.web.TestWebHDFS org.apache.hadoop.hdfs.server.datanode.TestBPOfferService org.apache.hadoop.hdfs.TestPersistBlocks +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1305//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1305//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          +1 lgtm

          Show
          Eli Collins added a comment - +1 lgtm
          Hide
          Tom White added a comment -

          Updated following YARN move. I successfully ran a job on a single-node cluster.

          Show
          Tom White added a comment - Updated following YARN move. I successfully ran a job on a single-node cluster.
          Hide
          Tom White added a comment -

          The test failure is unrelated.

          Show
          Tom White added a comment - The test failure is unrelated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12538208/HADOOP-8278.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 29 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples:

          org.apache.hadoop.fs.http.client.TestWebhdfsFileSystem

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1229//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1229//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12538208/HADOOP-8278.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 29 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples: org.apache.hadoop.fs.http.client.TestWebhdfsFileSystem +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1229//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1229//console This message is automatically generated.
          Hide
          Tom White added a comment -

          Synced with trunk. The two test failures pass locally. I also checked that I could run a job on a single node install.

          Show
          Tom White added a comment - Synced with trunk. The two test failures pass locally. I also checked that I could run a job on a single node install.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12531848/HADOOP-8278.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 20 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples:

          org.apache.hadoop.hdfs.TestDFSClientRetries
          org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1112//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1112//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531848/HADOOP-8278.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 20 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples: org.apache.hadoop.hdfs.TestDFSClientRetries org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1112//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1112//console This message is automatically generated.
          Hide
          Tom White added a comment -

          Thanks for taking a look, Alejandro. I've updated the patch to fix a few more dependencies in Common and HDFS, and also to add comments for some dependencies that the analysis doesn't find.

          Show
          Tom White added a comment - Thanks for taking a look, Alejandro. I've updated the patch to fix a few more dependencies in Common and HDFS, and also to add comments for some dependencies that the analysis doesn't find.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12529778/HADOOP-8278.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 javac. The applied patch generated 1999 javac compiler warnings (more than the trunk's current 1996 warnings).

          -1 javadoc. The javadoc tool appears to have generated 19 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples:

          org.apache.hadoop.http.TestHttpServer
          org.apache.hadoop.fs.viewfs.TestViewFsTrash

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//artifact/trunk/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12529778/HADOOP-8278.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. -1 javac. The applied patch generated 1999 javac compiler warnings (more than the trunk's current 1996 warnings). -1 javadoc. The javadoc tool appears to have generated 19 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples: org.apache.hadoop.http.TestHttpServer org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//artifact/trunk/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1038//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          forgot to mention, we should push this one one and then do MR as a separate JIRA.

          Show
          Alejandro Abdelnur added a comment - forgot to mention, we should push this one one and then do MR as a separate JIRA.
          Hide
          Alejandro Abdelnur added a comment -

          Tom, great job. I've rebased it to trunk (it was not applying cleanly) and I did a few tweaks to get a deployed cluster to work (added jersey-json back to hadoop-commons, it seems it is pulled via reflection and dependency:analyze misses that. Also set hadoop-auth as compile in httpfs).

          I'd suggest that for the dependencies that dependency:analyze misses we should add comments in the POMs.

          Show
          Alejandro Abdelnur added a comment - Tom, great job. I've rebased it to trunk (it was not applying cleanly) and I did a few tweaks to get a deployed cluster to work (added jersey-json back to hadoop-commons, it seems it is pulled via reflection and dependency:analyze misses that. Also set hadoop-auth as compile in httpfs). I'd suggest that for the dependencies that dependency:analyze misses we should add comments in the POMs.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526567/HADOOP-8278.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 javac. The applied patch generated 1937 javac compiler warnings (more than the trunk's current 1934 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples:

          org.apache.hadoop.http.TestHttpServer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//artifact/trunk/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526567/HADOOP-8278.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. -1 javac. The applied patch generated 1937 javac compiler warnings (more than the trunk's current 1934 warnings). +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-annotations hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal hadoop-mapreduce-project/hadoop-mapreduce-examples: org.apache.hadoop.http.TestHttpServer +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//artifact/trunk/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/987//console This message is automatically generated.
          Hide
          Tom White added a comment -

          Here's a patch to improve the dependencies for Common and HDFS.

          Show
          Tom White added a comment - Here's a patch to improve the dependencies for Common and HDFS.

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development