Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-4300

TransferFsImage.downloadEditsToStorage should use a tmp file for destination

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.1.0-beta
    • Component/s: None
    • Labels:
      None

      Description

      Currently, in TransferFsImage.downloadEditsToStorage, we download the edits file directly to its finalized path. So, if the transfer fails in the middle, a half-written file is left and cannot be distinguished from a correct file. So, future checkpoints by the 2NN will fail, since the file is truncated in the middle – but it won't ever download a good copy because it thinks it already has the proper file.

      1. hdfs-4300-5.patch
        17 kB
        Andrew Wang
      2. hdfs-4300-4.patch
        17 kB
        Aaron T. Myers
      3. hdfs-4300-3.patch
        17 kB
        Andrew Wang
      4. hdfs-4300-2.patch
        17 kB
        Andrew Wang
      5. hdfs-4300-1.patch
        10 kB
        Andrew Wang

        Issue Links

          Activity

          Hide
          Kihwal Lee added a comment -

          Will it be possible to somehow utilize the md5 file?

          Show
          Kihwal Lee added a comment - Will it be possible to somehow utilize the md5 file?
          Hide
          Todd Lipcon added a comment -

          We don't currently have an md5 file on the edits. We do detect that the transfer failed, but we don't properly use a temporary file. If we just write to a .tmp file and do the fsync/rename dance on successful transfer, we should be able to avoid the issue.

          Show
          Todd Lipcon added a comment - We don't currently have an md5 file on the edits. We do detect that the transfer failed, but we don't properly use a temporary file. If we just write to a .tmp file and do the fsync/rename dance on successful transfer, we should be able to avoid the issue.
          Hide
          Andrew Wang added a comment -

          Patch attached which does Todd's suggestion. I looked through the logs of the included test case, and saw that it correctly re-downloaded the truncated tmp file.

          Show
          Andrew Wang added a comment - Patch attached which does Todd's suggestion. I looked through the logs of the included test case, and saw that it correctly re-downloaded the truncated tmp file.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12577683/hdfs-4300-1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4205//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4205//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12577683/hdfs-4300-1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4205//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4205//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Hey Andrew. One question in a fault scenario: let's say it's trying to download some edits, and one of the dirs fails. This would leave the tmp files in place. Is it possible that then, in a future attempt at checkpointing, we might accidentally rename that tmp file into the final location?

          One potential fix for that would be to make the tmp file names use the current timestamp as a suffix, so that they aren't reused in later attempts.

          Show
          Todd Lipcon added a comment - Hey Andrew. One question in a fault scenario: let's say it's trying to download some edits, and one of the dirs fails. This would leave the tmp files in place. Is it possible that then, in a future attempt at checkpointing, we might accidentally rename that tmp file into the final location? One potential fix for that would be to make the tmp file names use the current timestamp as a suffix, so that they aren't reused in later attempts.
          Hide
          Andrew Wang added a comment -

          I think this is okay. A storage failure while downloading will blacklist the storage (I believe until the SNN is restarted), preventing rename. It also redownloads edits on a future attempt if the final edits file does not exist. So when we go to rename, we should only be iterating over valid storages with freshly downloaded files.

          Show
          Andrew Wang added a comment - I think this is okay. A storage failure while downloading will blacklist the storage (I believe until the SNN is restarted), preventing rename. It also redownloads edits on a future attempt if the final edits file does not exist. So when we go to rename, we should only be iterating over valid storages with freshly downloaded files.
          Hide
          Colin Patrick McCabe added a comment -

          I could be off base here, but I think there is some value in giving the temporary files unique names. Failed storage directories can come back (at least on the primary NN; haven't checked if this applies to the 2NN) if the correct configuration is set. Sure, everything "should" work correctly, but why take that chance when you can just use a unique file name?

          Show
          Colin Patrick McCabe added a comment - I could be off base here, but I think there is some value in giving the temporary files unique names. Failed storage directories can come back (at least on the primary NN; haven't checked if this applies to the 2NN) if the correct configuration is set. Sure, everything "should" work correctly, but why take that chance when you can just use a unique file name?
          Hide
          Andrew Wang added a comment -

          Thanks for the review Colin. This newest patch uses a temp filename based on getCurrentTimeMillis, which is nice for debugging and I think unique enough. We additionally do temp file cleanup on startup in SecondaryNameNode#initialize.

          Show
          Andrew Wang added a comment - Thanks for the review Colin. This newest patch uses a temp filename based on getCurrentTimeMillis , which is nice for debugging and I think unique enough. We additionally do temp file cleanup on startup in SecondaryNameNode#initialize .
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12581028/hdfs-4300-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12581028/hdfs-4300-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4338//console This message is automatically generated.
          Hide
          Andrew Wang added a comment -

          Fix the findbugs warning.

          Show
          Andrew Wang added a comment - Fix the findbugs warning.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12581085/hdfs-4300-3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.TestFsck

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4340//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4340//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12581085/hdfs-4300-3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestFsck +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4340//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4340//console This message is automatically generated.
          Hide
          Andrew Wang added a comment -

          test failure looks unrelated, shutdown failure in MiniDFSCluster.

          Show
          Andrew Wang added a comment - test failure looks unrelated, shutdown failure in MiniDFSCluster.
          Hide
          Aaron T. Myers added a comment -

          The most recent patch doesn't apply cleanly because of the commit of HDFS-4610. Here's a rebased version of the patch that's otherwise unchanged.

          The latest patch looks pretty good to me. Todd, does it look OK to you as well? If so, I'll go ahead and commit this rebased patch once Jenkins come back clean.

          Show
          Aaron T. Myers added a comment - The most recent patch doesn't apply cleanly because of the commit of HDFS-4610 . Here's a rebased version of the patch that's otherwise unchanged. The latest patch looks pretty good to me. Todd, does it look OK to you as well? If so, I'll go ahead and commit this rebased patch once Jenkins come back clean.
          Hide
          Todd Lipcon added a comment -

          A few small issues...

          In the test:

          +        new RandomAccessFile(tmpEdits[0], "rw").setLength(0);
          

          would leak the open file, right? This might cause the test to fail on Windows where, if I remember correctly, you can't delete a file which is open.

          Might also be worth a small refactor to extract a function for this repeated code:

          +        File[] tmpEdits = sd.getCurrentDir().listFiles(new FilenameFilter() {
          +          @Override
          +          public boolean accept(File dir, String name) {
          +            return name.startsWith(NameNodeFile.EDITS_TMP.getName());
          +          }
          +        });
          +        assertTrue(
          +            "Expected a single tmp edits file in directory " + sd.toString(),
          +            tmpEdits.length == 1);
          

          Seems like there's some debug printout left:

          +          System.out.println("deleteTempEdits: " + name);
          

          Otherwise looks good.

          Show
          Todd Lipcon added a comment - A few small issues... In the test: + new RandomAccessFile(tmpEdits[0], "rw" ).setLength(0); would leak the open file, right? This might cause the test to fail on Windows where, if I remember correctly, you can't delete a file which is open. Might also be worth a small refactor to extract a function for this repeated code: + File[] tmpEdits = sd.getCurrentDir().listFiles( new FilenameFilter() { + @Override + public boolean accept(File dir, String name) { + return name.startsWith(NameNodeFile.EDITS_TMP.getName()); + } + }); + assertTrue( + "Expected a single tmp edits file in directory " + sd.toString(), + tmpEdits.length == 1); Seems like there's some debug printout left: + System .out.println( "deleteTempEdits: " + name); Otherwise looks good.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12582687/hdfs-4300-4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4378//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4378//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12582687/hdfs-4300-4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4378//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4378//console This message is automatically generated.
          Hide
          Andrew Wang added a comment -

          Thanks for looking at this, Todd and ATM. Here's a new patch, also rebased, addressing Todd's latest comments.

          Show
          Andrew Wang added a comment - Thanks for looking at this, Todd and ATM. Here's a new patch, also rebased, addressing Todd's latest comments.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12582822/hdfs-4300-5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4384//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4384//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12582822/hdfs-4300-5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4384//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4384//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          +1, the latest patch looks good to me. I'm going to commit this momentarily.

          Show
          Aaron T. Myers added a comment - +1, the latest patch looks good to me. I'm going to commit this momentarily.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3747/)
          HDFS-4300. TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3747 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3747/ ) HDFS-4300 . TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Hide
          Aaron T. Myers added a comment -

          I've just committed this to trunk and branch-2.

          Thanks a lot for the contribution, Andrew, and thanks also to Todd for the reviews.

          Show
          Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Andrew, and thanks also to Todd for the reviews.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #209 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/209/)
          HDFS-4300. TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #209 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/209/ ) HDFS-4300 . TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1398 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1398/)
          HDFS-4300. TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987)

          Result = FAILURE
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1398 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1398/ ) HDFS-4300 . TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1425 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1425/)
          HDFS-4300. TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987)

          Result = SUCCESS
          atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1425 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1425/ ) HDFS-4300 . TransferFsImage.downloadEditsToStorage should use a tmp file for destination. Contributed by Andrew Wang. (Revision 1481987) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1481987 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CheckpointFaultInjector.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java

            People

            • Assignee:
              Andrew Wang
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development