Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3143

TestGetBlocks.testGetBlocks is failing

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: test
    • Labels:
      None

      Description

      TestGetBlocks.testGetBlocks is failing in the latest trunk/23 builds. Last good build was Mar 23rd.

      1. HDFS-3143.patch
        0.8 kB
        Arpit Gupta

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          I believe this test failure is due to HADOOP-8184.

          Show
          Aaron T. Myers added a comment - I believe this test failure is due to HADOOP-8184 .
          Hide
          Arpit Gupta added a comment -

          changed the assert to compare the class name rather than the comparing the message.

          Show
          Arpit Gupta added a comment - changed the assert to compare the class name rather than the comparing the message.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12520229/HDFS-3143.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
          org.apache.hadoop.hdfs.tools.TestDFSHAAdmin
          org.apache.hadoop.cli.TestHDFSCLI

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2107//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2107//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12520229/HDFS-3143.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade org.apache.hadoop.hdfs.tools.TestDFSHAAdmin org.apache.hadoop.cli.TestHDFSCLI +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2107//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2107//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          I'm sure that this patch will work to get the test passing, but I'm not sure that it's the most appropriate solution. Is it not possible to fix HADOOP-8184 so that the exception messages returned have the same data that they used to?

          Show
          Aaron T. Myers added a comment - I'm sure that this patch will work to get the test passing, but I'm not sure that it's the most appropriate solution. Is it not possible to fix HADOOP-8184 so that the exception messages returned have the same data that they used to?
          Hide
          Todd Lipcon added a comment -

          IMO it depends. The patch that caused this failure didn't make any note of semantic changes, so it seems a little strange to just change the test with no comment. But if this is in fact an improvement, it seems reasonable to fix the test. Could someone comment on what semantics actually changed? We used to prepend the exception class as part of the message of RemoteException, but now we don't?

          Show
          Todd Lipcon added a comment - IMO it depends. The patch that caused this failure didn't make any note of semantic changes, so it seems a little strange to just change the test with no comment. But if this is in fact an improvement, it seems reasonable to fix the test. Could someone comment on what semantics actually changed? We used to prepend the exception class as part of the message of RemoteException, but now we don't?
          Hide
          Aaron T. Myers added a comment -

          Great point, Todd. I'm also not entirely clear if the exception message change in HADOOP-8184 was an unintended side effect, or one of the intended effects of the JIRA.

          There is perhaps a question of compatibility here as well, since HADOOP-8184 changed the output of the `hadoop fs ...' CLI in the event of errors, as HDFS-3142 demonstrates. Should we be at all concerned about that? It doesn't seem like a big deal to me, but I'd like to hear others' opinions.

          Show
          Aaron T. Myers added a comment - Great point, Todd. I'm also not entirely clear if the exception message change in HADOOP-8184 was an unintended side effect, or one of the intended effects of the JIRA. There is perhaps a question of compatibility here as well, since HADOOP-8184 changed the output of the `hadoop fs ...' CLI in the event of errors, as HDFS-3142 demonstrates. Should we be at all concerned about that? It doesn't seem like a big deal to me, but I'd like to hear others' opinions.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Hi Aaron and Todd,

          The changes probably is not intentional but it is a good change since the class name should not be in the message. Exception messages are not APIs and so it can be changed anytime. Also, test should avoid depending on the message format. However, CLI error messages are a kind of public APIs. They should not be changed unless there are bugs.

          Since the new CLI re-write is not out yet and it is better not to have the class name in the message, I think we should keep the current message format and fix the tests. What do you think?

          Show
          Tsz Wo Nicholas Sze added a comment - Hi Aaron and Todd, The changes probably is not intentional but it is a good change since the class name should not be in the message. Exception messages are not APIs and so it can be changed anytime. Also, test should avoid depending on the message format. However, CLI error messages are a kind of public APIs. They should not be changed unless there are bugs. Since the new CLI re-write is not out yet and it is better not to have the class name in the message, I think we should keep the current message format and fix the tests. What do you think?
          Hide
          Todd Lipcon added a comment -

          Seems reasonable to me. We should mark HADOOP-8184 as an incompatible change and make sure this is noted in the release notes. Though they're not public APIs, there are cases in which I've seen software match against exception error messages – mostly because we have a very flat exception hierarchy in Hadoop. For example, HBase's FSHDFSUtils class does this in a few places to distinguish different cases of lease expiration.

          Show
          Todd Lipcon added a comment - Seems reasonable to me. We should mark HADOOP-8184 as an incompatible change and make sure this is noted in the release notes. Though they're not public APIs, there are cases in which I've seen software match against exception error messages – mostly because we have a very flat exception hierarchy in Hadoop. For example, HBase's FSHDFSUtils class does this in a few places to distinguish different cases of lease expiration.
          Hide
          Aaron T. Myers added a comment -

          Seems reasonable to me. We should mark HADOOP-8184 as an incompatible change and make sure this is noted in the release notes.

          +1

          Show
          Aaron T. Myers added a comment - Seems reasonable to me. We should mark HADOOP-8184 as an incompatible change and make sure this is noted in the release notes. +1
          Hide
          Aaron T. Myers added a comment -

          Also, +1 to this patch. I'll commit this shortly.

          Show
          Aaron T. Myers added a comment - Also, +1 to this patch. I'll commit this shortly.
          Hide
          Aaron T. Myers added a comment -

          I've just committed this to trunk and branch-2.

          Thanks a lot for the contribution, Arpit. Thanks for the discussion, Todd and Nicholas.

          Show
          Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Arpit. Thanks for the discussion, Todd and Nicholas.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2015 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2015/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2015 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2015/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1940 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1940/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1940 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1940/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1953 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1953/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = ABORTED
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1953 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1953/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1034 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1034/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1034 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1034/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #999 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/999/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #999 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/999/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1035 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1035/)
          HDFS-3143. TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1035 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1035/ ) HDFS-3143 . TestGetBlocks.testGetBlocks is failing. Contributed by Arpit Gupta. (Revision 1306542) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1306542 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java

            People

            • Assignee:
              Arpit Gupta
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development