Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3031

HA: Fix complete() and getAdditionalBlock() RPCs to be idempotent.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: ha
    • Labels:
      None

      Description

      I executed section 3.4 of Todd's HA test plan. https://issues.apache.org/jira/browse/HDFS-1623
      1. A large file upload is started.
      2. While the file is being uploaded, the administrator kills the first NN and performs a failover.
      3. After the file finishes being uploaded, it is verified for correct length and contents.

      For the test, I have a vm_template styx01:/home/schu/centos64-2-5.5.qcow2. styx01 hosted the active NN and styx02 hosted the standby NN.

      In the log files I attached, you can see that on styx01 I began file upload.
      hadoop fs -put centos64-2.5.5.qcow2

      After waiting several seconds, I kill -9'd the active NN on styx01 and manually failed over to the NN on styx02. I ran into exception below. (rest of the stacktrace in the attached file styx01_uploadLargeFile)

      12/02/29 14:12:52 WARN retry.RetryInvocationHandler: A failover has occurred since the start of this method invocation attempt.
      put: Failed on local exception: java.io.EOFException; Host Details : local host is: "styx01.sf.cloudera.com/172.29.5.192"; destination host is: ""styx01.sf.cloudera.com"\
      :12020;
      12/02/29 14:12:52 ERROR hdfs.DFSClient: Failed to close file /user/schu/centos64-2-5.5.qcow2.COPYING
      java.io.IOException: Failed on local exception: java.io.EOFException; Host Details : local host is: "styx01.sf.cloudera.com/172.29.5.192"; destination host is: ""styx01.\
      sf.cloudera.com":12020;
      at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
      at org.apache.hadoop.ipc.Client.call(Client.java:1145)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:188)
      at $Proxy9.addBlock(Unknown Source)
      at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:302)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
      at $Proxy10.addBlock(Unknown Source)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1097)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:973)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:455)
      Caused by: java.io.EOFException
      at java.io.DataInputStream.readInt(DataInputStream.java:375)
      at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:830)
      at org.apache.hadoop.ipc.Client$Connection.run(Client.java:762)

      1. hdfs-3031.txt
        27 kB
        Aaron T. Myers
      2. hdfs-3031.txt
        27 kB
        Aaron T. Myers
      3. hdfs-3031.txt
        27 kB
        Todd Lipcon
      4. hdfs-3031.txt
        21 kB
        Todd Lipcon
      5. hdfs-3031.txt
        22 kB
        Todd Lipcon
      6. hdfs-3031.txt
        13 kB
        Todd Lipcon
      7. styx01_killNNfailover
        4 kB
        Stephen Chu
      8. styx01_uploadLargeFile
        7 kB
        Stephen Chu

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          Seems like we have two problems:
          1) we don't notice that we've lost a connection to a NN until the next time we try to make an RPC
          2) addBlock isn't idempotent

          I think we may be able to address #1 in many cases by checking socket.isConnected() or something before making each RPC call. It's not guaranteed to work but makes it a tight race instead of a wide one.

          #2 could be addressed by augmenting addBlock() with a simple check: if the value passed for "previousBlock" is equal to the 2nd-from-last block of the file, and the last block of the file is under construction with 0 length, then we can simply re-return the last block.

          Show
          Todd Lipcon added a comment - Seems like we have two problems: 1) we don't notice that we've lost a connection to a NN until the next time we try to make an RPC 2) addBlock isn't idempotent I think we may be able to address #1 in many cases by checking socket.isConnected() or something before making each RPC call. It's not guaranteed to work but makes it a tight race instead of a wide one. #2 could be addressed by augmenting addBlock() with a simple check: if the value passed for "previousBlock" is equal to the 2nd-from-last block of the file, and the last block of the file is under construction with 0 length, then we can simply re-return the last block.
          Hide
          Aaron T. Myers added a comment -

          Converting to top-level issue with commit of HDFS-1623.

          Show
          Aaron T. Myers added a comment - Converting to top-level issue with commit of HDFS-1623 .
          Hide
          Todd Lipcon added a comment -

          This patch makes getAdditionalBlock() idempotent. I think it might still have some issues with append, perhaps - will see what tests it fails up on Jenkins

          Show
          Todd Lipcon added a comment - This patch makes getAdditionalBlock() idempotent. I think it might still have some issues with append, perhaps - will see what tests it fails up on Jenkins
          Hide
          Todd Lipcon added a comment -

          (aso need to make close() idempotent, I think, before we mark this JIRA resolved – just wanted to post the partial patch to see how it fares against the whole suite)

          Show
          Todd Lipcon added a comment - (aso need to make close() idempotent, I think, before we mark this JIRA resolved – just wanted to post the partial patch to see how it fares against the whole suite)
          Hide
          Todd Lipcon added a comment -

          New rev fixes the issue with append(): the problem is that the client doesn't send any previous block when appending to a file, when the append starts exactly at a block boundary.

          I attempted to make the client cleaner, but DFSOutputStream is a hairball. Without a substantial reworking of that, it was cleaner to do this on the server side.

          Show
          Todd Lipcon added a comment - New rev fixes the issue with append(): the problem is that the client doesn't send any previous block when appending to a file, when the append starts exactly at a block boundary. I attempted to make the client cleaner, but DFSOutputStream is a hairball. Without a substantial reworking of that, it was cleaner to do this on the server side.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526455/hdfs-3031.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2418//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526455/hdfs-3031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified test files. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2418//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Woops, my previously uploaded patch was accidentally against the auto-HA branch. New patch is against trunk (only a trivial difference in tests)

          Show
          Todd Lipcon added a comment - Woops, my previously uploaded patch was accidentally against the auto-HA branch. New patch is against trunk (only a trivial difference in tests)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526461/hdfs-3031.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2419//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2419//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526461/hdfs-3031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2419//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2419//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Patch looks pretty good to me, Todd. A few comments:

          1. Did you intend to leave that INFO log in DFSOutputStream? It seems to me like it should either be removed or lowered to debug or trace.
          2. Why make the changes in MiniDFSCluster? Those seem unrelated to this issue, and has the potential to cause unintended side effects in the tests, since some tests might be relying on the RPC behavior which this change cuts out of the call stack.

          I really like the new test testIdempotentAllocateBlock, and thanks for converting TestFileAppend3 to JUnit 4-style.

          Show
          Aaron T. Myers added a comment - Patch looks pretty good to me, Todd. A few comments: Did you intend to leave that INFO log in DFSOutputStream? It seems to me like it should either be removed or lowered to debug or trace. Why make the changes in MiniDFSCluster? Those seem unrelated to this issue, and has the potential to cause unintended side effects in the tests, since some tests might be relying on the RPC behavior which this change cuts out of the call stack. I really like the new test testIdempotentAllocateBlock, and thanks for converting TestFileAppend3 to JUnit 4-style.
          Hide
          Todd Lipcon added a comment -

          Attached patch implements idempotence on the complete() RPC as well.

          The approach is similar but simpler because the client reliably sends the last block as part of this IPC. Let me know if the new comment block isn't clear enough.

          I also removed the debug log message from DFSOutputStream that I accidentally left in.

          The reason for the test changes in MiniDFSCluster is that the cluster.transitionToActive() call to fail-back in doTestWriteOverFailoverWithDnFail was failing without it. The IPC client cache was holding a connection to the restarted node, so using the transitionToActive() RPC here threw an EOFException.

          I think if other tests want to explicitly test the RPC call, they should use the normal APIs like HAAdmin or FailoverController.

          Show
          Todd Lipcon added a comment - Attached patch implements idempotence on the complete() RPC as well. The approach is similar but simpler because the client reliably sends the last block as part of this IPC. Let me know if the new comment block isn't clear enough. I also removed the debug log message from DFSOutputStream that I accidentally left in. The reason for the test changes in MiniDFSCluster is that the cluster.transitionToActive() call to fail-back in doTestWriteOverFailoverWithDnFail was failing without it. The IPC client cache was holding a connection to the restarted node, so using the transitionToActive() RPC here threw an EOFException. I think if other tests want to explicitly test the RPC call, they should use the normal APIs like HAAdmin or FailoverController.
          Hide
          Aaron T. Myers added a comment -

          Latest patch looks great. +1 pending Jenkins.

          Show
          Aaron T. Myers added a comment - Latest patch looks great. +1 pending Jenkins.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526651/hdfs-3031.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2431//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2431//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526651/hdfs-3031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2431//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2431//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Looks like the MiniDFSCluster changes changed the expected exception stack in TestStateTransitionFailure.

          Here's an updated patch which is rebased on trunk with the following addition:

          diff --git hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java hadoop-hd
          index bf1ca52..3af78a5 100644
          --- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          @@ -67,8 +67,8 @@ public class TestStateTransitionFailure {
                   fail("Transitioned to active but should not have been able to.");
                 } catch (ServiceFailedException sfe) {
                   assertExceptionContains("Error encountered requiring NN shutdown. " +
          -            "Shutting down immediately.", sfe);
          -        LOG.info("got expected exception", sfe);
          +            "Shutting down immediately.", sfe.getCause());
          +        LOG.info("got expected exception", sfe.getCause());
                 }
                 verify(mockRuntime, times(1)).exit(anyInt());
               } finally {
          
          Show
          Aaron T. Myers added a comment - Looks like the MiniDFSCluster changes changed the expected exception stack in TestStateTransitionFailure. Here's an updated patch which is rebased on trunk with the following addition: diff --git hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java hadoop-hd index bf1ca52..3af78a5 100644 --- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java +++ hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java @@ -67,8 +67,8 @@ public class TestStateTransitionFailure { fail("Transitioned to active but should not have been able to."); } catch (ServiceFailedException sfe) { assertExceptionContains("Error encountered requiring NN shutdown. " + - "Shutting down immediately.", sfe); - LOG.info("got expected exception", sfe); + "Shutting down immediately.", sfe.getCause()); + LOG.info("got expected exception", sfe.getCause()); } verify(mockRuntime, times(1)).exit(anyInt()); } finally {
          Hide
          Aaron T. Myers added a comment -

          Forgot to grant license to ASF.

          Show
          Aaron T. Myers added a comment - Forgot to grant license to ASF.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526662/hdfs-3031.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2433//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2433//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526662/hdfs-3031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2433//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2433//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12526662/hdfs-3031.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2434//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2434//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526662/hdfs-3031.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2434//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2434//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Committed to branch-2 and trunk, thanks for fixing that test, Aaron.

          Show
          Todd Lipcon added a comment - Committed to branch-2 and trunk, thanks for fixing that test, Aaron.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2315 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2315/)
          HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2315 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2315/ ) HDFS-3031 . Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2241 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2241/)
          HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466)

          Result = SUCCESS
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2241 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2241/ ) HDFS-3031 . Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2258 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2258/)
          HDFS-3031. Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466)

          Result = ABORTED
          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2258 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2258/ ) HDFS-3031 . Fix complete() and getAdditionalBlock() RPCs to be idempotent. Contributed by Todd Lipcon. (Revision 1338466) Result = ABORTED todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1338466 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/Block.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestPipelinesFailover.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStateTransitionFailure.java
          Hide
          Suresh Srinivas added a comment -

          Changing the jira summary based on the text from CHANGES.txt

          Show
          Suresh Srinivas added a comment - Changing the jira summary based on the text from CHANGES.txt

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Stephen Chu
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development