Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9123

Copying from the root to a subdirectory should be forbidden

    Details

    • Hadoop Flags:
      Reviewed

      Description

      HDFS forbids copying from a directory to its subdirectory (e.g. hdfs dfs -cp /abc /abc/xyz) as otherwise it could cause infinite copying (/abc/xyz/xyz, /abc/xyz/xyz, /abc/xyz/xyz/xyz,... etc)

      However, the existing path validation fails if the source path is the root, causing infinite copying, until the disk space is filled up.

      1. HDFS-9123.004.patch
        2 kB
        Wei-Chiu Chuang
      2. HDFS-9123.003.patch
        2 kB
        Wei-Chiu Chuang
      3. HDFS-9123.002.patch
        4 kB
        Wei-Chiu Chuang
      4. HDFS-9123.001.patch
        4 kB
        Wei-Chiu Chuang

        Activity

        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        My first patch, along with a test case

        Show
        jojochuang Wei-Chiu Chuang added a comment - My first patch, along with a test case
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Fixed the buy by checking to see if the path is ended with a separator ('/').

        Show
        jojochuang Wei-Chiu Chuang added a comment - Fixed the buy by checking to see if the path is ended with a separator ('/').
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 17m 27s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 44s There were no new javac warning messages.
        +1 javadoc 10m 3s There were no new javadoc warning messages.
        +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 37s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 37s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 4m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 22m 14s Tests failed in hadoop-common.
        +1 hdfs tests 163m 42s Tests passed in hadoop-hdfs.
            229m 44s  



        Reason Tests
        Failed unit tests hadoop.ipc.TestRPC



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761764/HDFS-9123.001.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / cc2b473
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12613/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12613/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12613/testReport/
        Java 1.7.0_55
        uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12613/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 27s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 44s There were no new javac warning messages. +1 javadoc 10m 3s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 37s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 37s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 14s Tests failed in hadoop-common. +1 hdfs tests 163m 42s Tests passed in hadoop-hdfs.     229m 44s   Reason Tests Failed unit tests hadoop.ipc.TestRPC Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761764/HDFS-9123.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / cc2b473 hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12613/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12613/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12613/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12613/console This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        My patch did not touch IPC at all. I ran all tests locally but did not see the same failure again.

        Show
        jojochuang Wei-Chiu Chuang added a comment - My patch did not touch IPC at all. I ran all tests locally but did not see the same failure again.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Wei-Chiu Chuang,

        Thanks for reporting the issue and the patch. It looks good in general and I have some minor comments:

        1. Suggest to change

             String pathEnd = "";
              if(!srcPath.toString().endsWith(Path.SEPARATOR)){
                pathEnd = Path.SEPARATOR;
              }
              if(dstPath.startsWith(srcPath+pathEnd)) {
        

        to

              if (!srcPath.endsWith(Path.SEPARATOR)) {
                srcPath += Path.SEPARATOR;
              }
              if(dstPath.startsWith(srcPath)) {
        

        2. The test looks good, i can see that it report the "cp: `/' to `/test/': is a subdirectory of itself" message with the fix. However, if I remove the fix and run the same test, it's supposed to fail but it doesn't. Would you please look into?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Wei-Chiu Chuang , Thanks for reporting the issue and the patch. It looks good in general and I have some minor comments: 1. Suggest to change String pathEnd = ""; if (!srcPath.toString().endsWith(Path.SEPARATOR)){ pathEnd = Path.SEPARATOR; } if (dstPath.startsWith(srcPath+pathEnd)) { to if (!srcPath.endsWith(Path.SEPARATOR)) { srcPath += Path.SEPARATOR; } if (dstPath.startsWith(srcPath)) { 2. The test looks good, i can see that it report the "cp: `/' to `/test/': is a subdirectory of itself" message with the fix. However, if I remove the fix and run the same test, it's supposed to fail but it doesn't. Would you please look into? Thanks.
        Hide
        yzhangal Yongjun Zhang added a comment -

        BTW, I noticed that your patch rev 2 is not correctly generated.

        Show
        yzhangal Yongjun Zhang added a comment - BTW, I noticed that your patch rev 2 is not correctly generated.
        Hide
        andreina J.Andreina added a comment -

        Thanks Wei-Chiu Chuang . That was a good catch. I have few comments .

        if the source path is ended with a '/' path separator, the existing validation for sub-directories fails

        As I have verified, issue of copying indefinitely, exist only if src is just root ('/"). It will not happen for any other paths ending with '/'.

        	> ./hdfs dfs -cp /abc/ /abc/abcdef
        	cp: `/abc' to `/abc/abcdef/abc': is a subdirectory of itself
        

        So, I feel since this is only for src is root, validation can be updated addressing this special case.
        I think below code would fix it correctly

        -      String srcPath = src.fs.makeQualified(src.path).toString();
        -      String dstPath = dst.fs.makeQualified(target.path).toString();
        +      Path srcPath = src.fs.makeQualified(src.path);
        +      Path dstPath = dst.fs.makeQualified(target.path);
               if (dstPath.equals(srcPath)) {
                 PathIOException e = new PathIOException(src.toString(),
                     "are identical");
                 e.setTargetPath(dstPath.toString());
                 throw e;
               }
        -      if (dstPath.startsWith(srcPath+Path.SEPARATOR)) {
        +      if (srcPath.isRoot() || dstPath.toString().startsWith(srcPath.toString() + Path.SEPARATOR)) {
        

        Also it would be good if you can update the jira title and summary to reflect the special case

        Show
        andreina J.Andreina added a comment - Thanks Wei-Chiu Chuang . That was a good catch. I have few comments . if the source path is ended with a '/' path separator, the existing validation for sub-directories fails As I have verified, issue of copying indefinitely, exist only if src is just root ('/"). It will not happen for any other paths ending with '/'. > ./hdfs dfs -cp /abc/ /abc/abcdef cp: `/abc' to `/abc/abcdef/abc': is a subdirectory of itself So, I feel since this is only for src is root, validation can be updated addressing this special case. I think below code would fix it correctly - String srcPath = src.fs.makeQualified(src.path).toString(); - String dstPath = dst.fs.makeQualified(target.path).toString(); + Path srcPath = src.fs.makeQualified(src.path); + Path dstPath = dst.fs.makeQualified(target.path); if (dstPath.equals(srcPath)) { PathIOException e = new PathIOException(src.toString(), "are identical" ); e.setTargetPath(dstPath.toString()); throw e; } - if (dstPath.startsWith(srcPath+Path.SEPARATOR)) { + if (srcPath.isRoot() || dstPath.toString().startsWith(srcPath.toString() + Path.SEPARATOR)) { Also it would be good if you can update the jira title and summary to reflect the special case
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 22m 12s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 58s There were no new javac warning messages.
        +1 javadoc 10m 49s There were no new javadoc warning messages.
        +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 58s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 42s mvn install still works.
        +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse.
        +1 findbugs 5m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 24m 1s Tests failed in hadoop-common.
        -1 hdfs tests 120m 56s Tests failed in hadoop-hdfs.
            196m 50s  



        Reason Tests
        Failed unit tests hadoop.security.ssl.TestReloadingX509TrustManager
          hadoop.hdfs.TestReplaceDatanodeOnFailure
          hadoop.hdfs.web.TestWebHDFSOAuth2
        Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAMetrics



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761806/HDFS-9123.002.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 7c5c099
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12616/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12616/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12616/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12616/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 22m 12s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 49s There were no new javadoc warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 58s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 42s mvn install still works. +1 eclipse:eclipse 0m 41s The patch built with eclipse:eclipse. +1 findbugs 5m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 24m 1s Tests failed in hadoop-common. -1 hdfs tests 120m 56s Tests failed in hadoop-hdfs.     196m 50s   Reason Tests Failed unit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.web.TestWebHDFSOAuth2 Timed out tests org.apache.hadoop.hdfs.server.namenode.ha.TestHAMetrics Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761806/HDFS-9123.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7c5c099 hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12616/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12616/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12616/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12616/console This message was automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        should the tests really downgrade exceptions to logs? I'd expect them to signal failure and be rethrown

                try {
        	          val = shell.run(args1);
        	        } catch (Exception e) {
        	          System.err.println("Exception raised from DFSShell.run " +
        	                  e.getLocalizedMessage());
        	        }
        
        Show
        stevel@apache.org Steve Loughran added a comment - should the tests really downgrade exceptions to logs? I'd expect them to signal failure and be rethrown try { val = shell.run(args1); } catch (Exception e) { System .err.println( "Exception raised from DFSShell.run " + e.getLocalizedMessage()); }
        Hide
        andreina J.Andreina added a comment -

        @Steve Loughran, that was a good point.

        In this testcase, test will fail by the next assertion on return code if exception is not thrown.

        	try {
                  val = shell.run(args1);
                } catch (Exception e) {
                  System.err.println("Exception raised from DFSShell.run " +
                          e.getLocalizedMessage());
                }
                assertEquals(1, val);
        

        And when i checked , other similar tests also uses the same pattern in this test class.

        Show
        andreina J.Andreina added a comment - @Steve Loughran, that was a good point. In this testcase, test will fail by the next assertion on return code if exception is not thrown. try { val = shell.run(args1); } catch (Exception e) { System .err.println( "Exception raised from DFSShell.run " + e.getLocalizedMessage()); } assertEquals(1, val); And when i checked , other similar tests also uses the same pattern in this test class.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        J.Andreina Thanks for pointing out. Yes indeed it looks like the case, because when makeQualified() is called, the path is normalized and trailing slashes are removed except for the root. I am going to update the title/description of this jira.

        Show
        jojochuang Wei-Chiu Chuang added a comment - J.Andreina Thanks for pointing out. Yes indeed it looks like the case, because when makeQualified() is called, the path is normalized and trailing slashes are removed except for the root. I am going to update the title/description of this jira.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        However, the fact that the source path is the root, does not imply the destination path must be its sub-directory. Correct? I mean they do not necessarily belong to the same file system.

        Show
        jojochuang Wei-Chiu Chuang added a comment - However, the fact that the source path is the root, does not imply the destination path must be its sub-directory. Correct? I mean they do not necessarily belong to the same file system.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi Guys, thanks for the review and comments.

        However, the fact that the source path is the root, does not imply the destination path must be its sub-directory. Correct? I mean they do not necessarily belong to the same file system.

        Correct.

        About

        should the tests really downgrade exceptions to logs? I'd expect them to signal failure and be rethrown

        Good point. Since all the tests in this test file are coded like that, I think we can create a new jira to fix all of them.
        For HDFS-9123, I'd suggest that we should make the newly added test fail when without the fix (it currently succeeds as I mentioned earlier), and make it succeed when with the fix.

        Do you guys agree?

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi Guys, thanks for the review and comments. However, the fact that the source path is the root, does not imply the destination path must be its sub-directory. Correct? I mean they do not necessarily belong to the same file system. Correct. About should the tests really downgrade exceptions to logs? I'd expect them to signal failure and be rethrown Good point. Since all the tests in this test file are coded like that, I think we can create a new jira to fix all of them. For HDFS-9123 , I'd suggest that we should make the newly added test fail when without the fix (it currently succeeds as I mentioned earlier), and make it succeed when with the fix. Do you guys agree? Thanks.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Updated the code to deal with this bug as a special case when the source is the root.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Updated the code to deal with this bug as a special case when the source is the root.
        Hide
        liuml07 Mingliang Liu added a comment -

        This is a good point. I supposed this is the case for distcp instead of cp?

        Show
        liuml07 Mingliang Liu added a comment - This is a good point. I supposed this is the case for distcp instead of cp ?
        Hide
        liuml07 Mingliang Liu added a comment -

        The patch looks good to me.

        Show
        liuml07 Mingliang Liu added a comment - The patch looks good to me.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hello Mingliang Liu,

        Thanks for the comments.

        This is a good point. I supposed this is the case for distcp instead of cp?

        it's for "fs-cp", not distcp here. It also supports copying to different file systems.

        Hi Wei-Chiu Chuang,

        Thanks for the new rev.

        Would you please address comment # 1 at
        https://issues.apache.org/jira/browse/HDFS-9123?focusedCommentId=14904029&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14904029
        ?
        About my comment #2, I studied the code a bit, and found that FsShell simply absort all exceptions, and return status code, the client code has no way to tell what's the exact failure reason if a command failed. So I'm ok that comment #2 is not addressed. At least, the new test can show that copying from root to child fail the same way as copying from any parent to child.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hello Mingliang Liu , Thanks for the comments. This is a good point. I supposed this is the case for distcp instead of cp? it's for "fs-cp", not distcp here. It also supports copying to different file systems. Hi Wei-Chiu Chuang , Thanks for the new rev. Would you please address comment # 1 at https://issues.apache.org/jira/browse/HDFS-9123?focusedCommentId=14904029&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14904029 ? About my comment #2, I studied the code a bit, and found that FsShell simply absort all exceptions, and return status code, the client code has no way to tell what's the exact failure reason if a command failed. So I'm ok that comment #2 is not addressed. At least, the new test can show that copying from root to child fail the same way as copying from any parent to child. Thanks.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Rev 3.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Rev 3.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for Yongjun's comment. Here's rev4 that addresses your comment #1.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for Yongjun's comment. Here's rev4 that addresses your comment #1.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Thanks for every one's participation. I am going to submit rev4 as patch.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Thanks for every one's participation. I am going to submit rev4 as patch.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Agree with Yongjun's comment #2. The shell currently is not designed to throw internal exceptions. It would require another lira to make a better test design though.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Agree with Yongjun's comment #2. The shell currently is not designed to throw internal exceptions. It would require another lira to make a better test design though.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Wei-Chiu for addressing my comments. rev4 looks good to me. +1 pending jenkins.

        Thanks other folks for the review. I plan to commit this tomorrow. If you have further comment on it, would you please kindly provide asap? Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Wei-Chiu for addressing my comments. rev4 looks good to me. +1 pending jenkins. Thanks other folks for the review. I plan to commit this tomorrow. If you have further comment on it, would you please kindly provide asap? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 20m 6s Findbugs (version ) appears to be broken on trunk.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 10m 43s There were no new javac warning messages.
        +1 javadoc 13m 47s There were no new javadoc warning messages.
        +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 4s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 2m 4s mvn install still works.
        +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse.
        +1 findbugs 6m 11s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 27m 50s Tests failed in hadoop-common.
        -1 hdfs tests 116m 14s Tests failed in hadoop-hdfs.
            200m 21s  



        Reason Tests
        Failed unit tests hadoop.ipc.TestDecayRpcScheduler
          hadoop.ipc.TestRPC
          hadoop.fs.TestLocalFsFCStatistics
          hadoop.metrics2.impl.TestMetricsSystemImpl
          hadoop.hdfs.server.blockmanagement.TestBlockManager
        Timed out tests org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761942/HDFS-9123.003.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 692d51c
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12630/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12630/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12630/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12630/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 20m 6s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 10m 43s There were no new javac warning messages. +1 javadoc 13m 47s There were no new javadoc warning messages. +1 release audit 0m 33s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 4s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 2m 4s mvn install still works. +1 eclipse:eclipse 0m 42s The patch built with eclipse:eclipse. +1 findbugs 6m 11s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 27m 50s Tests failed in hadoop-common. -1 hdfs tests 116m 14s Tests failed in hadoop-hdfs.     200m 21s   Reason Tests Failed unit tests hadoop.ipc.TestDecayRpcScheduler   hadoop.ipc.TestRPC   hadoop.fs.TestLocalFsFCStatistics   hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.hdfs.server.blockmanagement.TestBlockManager Timed out tests org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761942/HDFS-9123.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 692d51c hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12630/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12630/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12630/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12630/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 20m 4s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 59s There were no new javac warning messages.
        +1 javadoc 10m 11s There were no new javadoc warning messages.
        +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 36s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 22m 43s Tests passed in hadoop-common.
        -1 hdfs tests 163m 13s Tests failed in hadoop-hdfs.
            233m 48s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestDNFencing



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12761952/HDFS-9123.004.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / b3f6b64
        hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12633/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12633/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12633/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12633/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 20m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 59s There were no new javac warning messages. +1 javadoc 10m 11s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 36s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 22m 43s Tests passed in hadoop-common. -1 hdfs tests 163m 13s Tests failed in hadoop-hdfs.     233m 48s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestDNFencing Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761952/HDFS-9123.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / b3f6b64 hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12633/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12633/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12633/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12633/console This message was automatically generated.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thanks Wei-Chiu Chuang for the contribution and all for the review, I committed to trunk and branch-2.

        Show
        yzhangal Yongjun Zhang added a comment - Thanks Wei-Chiu Chuang for the contribution and all for the review, I committed to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #439 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/439/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #439 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/439/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8518 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8518/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8518 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8518/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1179 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1179/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1179 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1179/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #446 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/446/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #446 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/446/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2357 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2357/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2357 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2357/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #417 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/417/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #417 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/417/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2384/)
        HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2384 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2384/ ) HDFS-9123 . Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) (yzhang: rev 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            jojochuang Wei-Chiu Chuang
            Reporter:
            jojochuang Wei-Chiu Chuang
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development