Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Got this exception when running distcp -diff with relative paths:

      $ hadoop distcp -update -diff s1 s2 d1 d2
      16/03/25 09:45:40 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=true, deleteMissing=false, ignoreFailures=false, maxMaps=20, sslConfigurationFile='null', copyStrategy='uniformsize', sourceFileListing=null, sourcePaths=[d1], targetPath=d2, targetPathExists=true, preserveRawXattrs=false, filtersFile='null'}
      16/03/25 09:45:40 INFO client.RMProxy: Connecting to ResourceManager at jzhuge-balancer-1.vpc.cloudera.com/172.26.21.70:8032
      16/03/25 09:45:41 ERROR tools.DistCp: Exception encountered 
      java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: hdfs://jzhuge-balancer-1.vpc.cloudera.com:8020./d1/.snapshot/s2
      	at org.apache.hadoop.fs.Path.initialize(Path.java:206)
      	at org.apache.hadoop.fs.Path.<init>(Path.java:197)
      	at org.apache.hadoop.tools.SimpleCopyListing.getPathWithSchemeAndAuthority(SimpleCopyListing.java:193)
      	at org.apache.hadoop.tools.SimpleCopyListing.addToFileListing(SimpleCopyListing.java:202)
      	at org.apache.hadoop.tools.SimpleCopyListing.doBuildListingWithSnapshotDiff(SimpleCopyListing.java:243)
      	at org.apache.hadoop.tools.SimpleCopyListing.doBuildListing(SimpleCopyListing.java:172)
      	at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
      	at org.apache.hadoop.tools.DistCp.createInputFileListingWithDiff(DistCp.java:388)
      	at org.apache.hadoop.tools.DistCp.execute(DistCp.java:164)
      	at org.apache.hadoop.tools.DistCp.run(DistCp.java:123)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.tools.DistCp.main(DistCp.java:436)
      Caused by: java.net.URISyntaxException: Relative path in absolute URI: hdfs://jzhuge-balancer-1.vpc.cloudera.com:8020./d1/.snapshot/s2
      	at java.net.URI.checkPath(URI.java:1804)
      	at java.net.URI.<init>(URI.java:752)
      	at org.apache.hadoop.fs.Path.initialize(Path.java:203)
      	... 11 more
      

      But theses commands worked:

      • Absolute path: hadoop distcp -update -diff s1 s2 /user/systest/d1 /user/systest/d2
      • No -diff: hadoop distcp -update d1 d2

      However, everything was fine when I ran hadoop distcp -update -diff s1 s2 d1 d2 again. I am not sure the problem only exists with option -diff. Trying to reproduce.

      1. HDFS-10216.1.patch
        0.7 kB
        Takashi Ohnishi
      2. HDFS-10216.2.patch
        2 kB
        Takashi Ohnishi
      3. HDFS-10216.3.patch
        2 kB
        Takashi Ohnishi
      4. HDFS-10216.4.patch
        2 kB
        Takashi Ohnishi

        Activity

        Hide
        jzhuge John Zhuge added a comment -

        Did hit the exception twice on 2 different nodes when running hadoop distcp -update -diff s1 s2 d1 d2 initially.

        Show
        jzhuge John Zhuge added a comment - Did hit the exception twice on 2 different nodes when running hadoop distcp -update -diff s1 s2 d1 d2 initially.
        Hide
        jingzhao Jing Zhao added a comment -

        Looks like getPathWithSchemeAndAuthority does not correctly handle relative path when creating the path:

            }
        
            return new Path(scheme, authority, path.toUri().getPath());
        

        Do you want to provide a patch to fix this, John Zhuge?

        Show
        jingzhao Jing Zhao added a comment - Looks like getPathWithSchemeAndAuthority does not correctly handle relative path when creating the path: } return new Path(scheme, authority, path.toUri().getPath()); Do you want to provide a patch to fix this, John Zhuge ?
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        I hit this, too.

        May I try to create a patch for this?

        Show
        bwtakacy Takashi Ohnishi added a comment - I hit this, too. May I try to create a patch for this?
        Hide
        jzhuge John Zhuge added a comment -

        Please go ahead.

        Show
        jzhuge John Zhuge added a comment - Please go ahead.
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thanks!

        Show
        bwtakacy Takashi Ohnishi added a comment - Thanks!
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Attached patch.

        Show
        bwtakacy Takashi Ohnishi added a comment - Attached patch.
        Hide
        jzhuge John Zhuge added a comment -

        Great job with the patch Takashi Ohnishi.

        I assume the ad-hoc test passed with the patch. Is there any unit test that can reproduce this problem? If not, should we add a unit test?

        Should we run related unit tests to exercise this code path?

        Show
        jzhuge John Zhuge added a comment - Great job with the patch Takashi Ohnishi . I assume the ad-hoc test passed with the patch. Is there any unit test that can reproduce this problem? If not, should we add a unit test? Should we run related unit tests to exercise this code path?
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thank you for commenting, John Zhuge.
        I tested the patch locally and this remove the failure of relative path in my case.
        I also confirmed this patch does not break the existed tests of distcp.
        But, there seems to be no exact test reproducing this problem.
        I will try to add a unit test.

        Show
        bwtakacy Takashi Ohnishi added a comment - Thank you for commenting, John Zhuge . I tested the patch locally and this remove the failure of relative path in my case. I also confirmed this patch does not break the existed tests of distcp. But, there seems to be no exact test reproducing this problem. I will try to add a unit test.
        Hide
        jzhuge John Zhuge added a comment -

        Thanks for enhancing the unit tests which will definitely improve the code quality and prevent future regression.

        Show
        jzhuge John Zhuge added a comment - Thanks for enhancing the unit tests which will definitely improve the code quality and prevent future regression.
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Attached the updated patch.
        I have added a unit test for relative path problem in TestDistCpSync.

        Show
        bwtakacy Takashi Ohnishi added a comment - Attached the updated patch. I have added a unit test for relative path problem in TestDistCpSync.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 57s trunk passed
        +1 compile 0m 16s trunk passed with JDK v1.8.0_77
        +1 compile 0m 17s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 0m 29s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 17s the patch passed
        +1 compile 0m 13s the patch passed with JDK v1.8.0_77
        +1 javac 0m 13s the patch passed
        +1 compile 0m 15s the patch passed with JDK v1.7.0_95
        +1 javac 0m 15s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 38s the patch passed
        +1 javadoc 0m 9s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
        +1 unit 7m 49s hadoop-distcp in the patch passed with JDK v1.8.0_77.
        +1 unit 6m 54s hadoop-distcp in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        27m 56s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798508/HDFS-10216.2.patch
        JIRA Issue HDFS-10216
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ea3cd1104fd7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 903428b
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15149/testReport/
        modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15149/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 57s trunk passed +1 compile 0m 16s trunk passed with JDK v1.8.0_77 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 29s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 17s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_77 +1 javac 0m 13s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_95 +1 javac 0m 15s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 9s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 7m 49s hadoop-distcp in the patch passed with JDK v1.8.0_77. +1 unit 6m 54s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 27m 56s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798508/HDFS-10216.2.patch JIRA Issue HDFS-10216 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ea3cd1104fd7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 903428b Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15149/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15149/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        Great job. Just minor nitpicks:

        • TestDistCpSync.java
          1. 683: foo is unused
          2. 697: use dfs.getWorkingDirectory(), e.g.:
                Path sourcePath = new Path(dfs.getWorkingDirectory(), "source");
            
          3. 708: use target.toString() instead of "/target".
          4. 710-714: try block is not necessary; junit will catch and print full stacktrace.
        • In general, avoid hard-coded "/", use Path.SEPARATOR instead.
        Show
        jzhuge John Zhuge added a comment - Great job. Just minor nitpicks: TestDistCpSync.java 683: foo is unused 697: use dfs.getWorkingDirectory() , e.g.: Path sourcePath = new Path(dfs.getWorkingDirectory(), "source" ); 708: use target.toString() instead of "/target". 710-714: try block is not necessary; junit will catch and print full stacktrace. In general, avoid hard-coded "/", use Path.SEPARATOR instead.
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thank you, John Zhuge for helpful reviewing !

        I have updated the patch with your advice. Please see the attached one.

        1. 683: foo is unused

        Oops. I want to create f3 under foo dir, but not did so.

        2. 697: use dfs.getWorkingDirectory(), e.g.:

            Path sourcePath = new Path(dfs.getWorkingDirectory(), "source");
        

        Ah, that's a very smart.

        3. 708: use target.toString() instead of "/target".
        4. 710-714: try block is not necessary; junit will catch and print full stacktrace.
        In general, avoid hard-coded "/", use Path.SEPARATOR instead.

        I have fixed them.

        Show
        bwtakacy Takashi Ohnishi added a comment - Thank you, John Zhuge for helpful reviewing ! I have updated the patch with your advice. Please see the attached one. 1. 683: foo is unused Oops. I want to create f3 under foo dir, but not did so. 2. 697: use dfs.getWorkingDirectory(), e.g.: Path sourcePath = new Path(dfs.getWorkingDirectory(), "source" ); Ah, that's a very smart. 3. 708: use target.toString() instead of "/target". 4. 710-714: try block is not necessary; junit will catch and print full stacktrace. In general, avoid hard-coded "/", use Path.SEPARATOR instead. I have fixed them.
        Hide
        jzhuge John Zhuge added a comment -

        Very nice. Consider reviewed after 2 tiny updates:

        • Line 709 is too long
        • Line 710-711 can be merged into 1 line: new DistCp(...).execute()
        Show
        jzhuge John Zhuge added a comment - Very nice. Consider reviewed after 2 tiny updates: Line 709 is too long Line 710-711 can be merged into 1 line: new DistCp(...).execute()
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 45s trunk passed
        +1 compile 0m 13s trunk passed with JDK v1.8.0_77
        +1 compile 0m 17s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 13s trunk passed
        +1 mvnsite 0m 23s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 0m 30s trunk passed
        +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_77
        +1 javac 0m 12s the patch passed
        +1 compile 0m 15s the patch passed with JDK v1.7.0_95
        +1 javac 0m 15s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 38s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
        +1 unit 8m 1s hadoop-distcp in the patch passed with JDK v1.8.0_77.
        +1 unit 7m 11s hadoop-distcp in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 19s Patch does not generate ASF License warnings.
        28m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798629/HDFS-10216.3.patch
        JIRA Issue HDFS-10216
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 3fa88677d830 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 27b131e
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15157/testReport/
        modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15157/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 45s trunk passed +1 compile 0m 13s trunk passed with JDK v1.8.0_77 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_77 +1 javac 0m 12s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_95 +1 javac 0m 15s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 38s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 8m 1s hadoop-distcp in the patch passed with JDK v1.8.0_77. +1 unit 7m 11s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s Patch does not generate ASF License warnings. 28m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798629/HDFS-10216.3.patch JIRA Issue HDFS-10216 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 3fa88677d830 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 27b131e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15157/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15157/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        bwtakacy Takashi Ohnishi added a comment -
        • Line 709 is too long
        • Line 710-711 can be merged into 1 line: new DistCp(...).execute()

        All right. I have fixed them in the v4 patch.
        And, I have added a verification the result of distcp.

        Show
        bwtakacy Takashi Ohnishi added a comment - Line 709 is too long Line 710-711 can be merged into 1 line: new DistCp(...).execute() All right. I have fixed them in the v4 patch. And, I have added a verification the result of distcp.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 9s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 4s trunk passed
        +1 compile 0m 14s trunk passed with JDK v1.8.0_77
        +1 compile 0m 17s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 14s trunk passed
        +1 mvnsite 0m 22s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 28s trunk passed
        +1 javadoc 0m 12s trunk passed with JDK v1.8.0_77
        +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 18s the patch passed
        +1 compile 0m 12s the patch passed with JDK v1.8.0_77
        +1 javac 0m 12s the patch passed
        +1 compile 0m 15s the patch passed with JDK v1.7.0_95
        +1 javac 0m 15s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 20s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 39s the patch passed
        +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77
        +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95
        +1 unit 8m 16s hadoop-distcp in the patch passed with JDK v1.8.0_77.
        +1 unit 7m 39s hadoop-distcp in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        29m 4s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798718/HDFS-10216.4.patch
        JIRA Issue HDFS-10216
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b748750ed614 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / df18b6e9
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15163/testReport/
        modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15163/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 4s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_77 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_77 +1 javac 0m 12s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_95 +1 javac 0m 15s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_95 +1 unit 8m 16s hadoop-distcp in the patch passed with JDK v1.8.0_77. +1 unit 7m 39s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 29m 4s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798718/HDFS-10216.4.patch JIRA Issue HDFS-10216 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b748750ed614 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / df18b6e9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15163/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15163/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        +1 LGTM

        Show
        jzhuge John Zhuge added a comment - +1 LGTM
        Hide
        jingzhao Jing Zhao added a comment -

        +1. I've committed this to trunk and branch-2. Thanks for the fix, Takashi Ohnishi. Thanks for reporting the issue and reviewing the patch, John Zhuge.

        Show
        jingzhao Jing Zhao added a comment - +1. I've committed this to trunk and branch-2. Thanks for the fix, Takashi Ohnishi . Thanks for reporting the issue and reviewing the patch, John Zhuge .
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9612 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9612/)
        HDFS-10216. Distcp -diff throws exception when handling relative path. (jing9: rev 404f57f328b00a42ec8b952ad08cd7a80207c7f2)

        • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java
        • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9612 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9612/ ) HDFS-10216 . Distcp -diff throws exception when handling relative path. (jing9: rev 404f57f328b00a42ec8b952ad08cd7a80207c7f2) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpSync.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thank you Jing Zhao for committing !

        Thank you John Zhuge for helpful reviewing !!

        Show
        bwtakacy Takashi Ohnishi added a comment - Thank you Jing Zhao for committing ! Thank you John Zhuge for helpful reviewing !!
        Hide
        yzhangal Yongjun Zhang added a comment -

        Thank you guys for the work here. I just cherry-picked it to branch-2.8.

        Show
        yzhangal Yongjun Zhang added a comment - Thank you guys for the work here. I just cherry-picked it to branch-2.8.
        Hide
        jzhuge John Zhuge added a comment -

        Thanks Yongjun Zhang.

        Show
        jzhuge John Zhuge added a comment - Thanks Yongjun Zhang .

          People

          • Assignee:
            bwtakacy Takashi Ohnishi
            Reporter:
            jzhuge John Zhuge
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development