Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13034

Log message about input options in distcp lacks some items

        Details

        • Type: Improvement
        • Status: Resolved
        • Priority: Minor
        • Resolution: Fixed
        • Affects Version/s: 3.0.0-alpha1
        • Fix Version/s: 2.9.0, 3.0.0-alpha1
        • Component/s: tools/distcp
        • Labels:
          None

          Description

          The log message in running distcp does not show some options, i.e. append, useDiff and snapshot names.

          16/04/18 21:57:36 INFO tools.DistCp: Input Options: DistCpOptions{atomicCommit=false, syncFolder=true, deleteMissing=false, ignoreFailures=false, overwrite=false, skipCRC=false, blocking=true, numListstatusThreads=0, maxMaps=20, mapBandwidth=100.0, copyStrategy='uniformsize', preserveStatus=[], preserveRawXattrs=false, atomicWorkPath=null, logPath=null, sourceFileListing=null, sourcePaths=[/user/hadoop/source], targetPath=/user/hadoop/target, targetPathExists=true, filtersFile='null'}

          I think that this message is useful for debugging and so it is better to add the lacked options.

          1. Text File
            HADOOP-13034.1.patch
            2 kB
            Takashi Ohnishi

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            bwtakacy Takashi Ohnishi added a comment -

            Attached patch.

            Show
            bwtakacy Takashi Ohnishi added a comment - Attached patch.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 15s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
            +1 mvninstall 9m 41s trunk passed
            +1 compile 0m 28s trunk passed with JDK v1.8.0_77
            +1 compile 0m 25s trunk passed with JDK v1.7.0_95
            +1 checkstyle 0m 22s trunk passed
            +1 mvnsite 0m 32s trunk passed
            +1 mvneclipse 0m 21s trunk passed
            +1 findbugs 0m 41s trunk passed
            +1 javadoc 0m 24s trunk passed with JDK v1.8.0_77
            +1 javadoc 0m 22s trunk passed with JDK v1.7.0_95
            +1 mvninstall 0m 24s the patch passed
            +1 compile 0m 23s the patch passed with JDK v1.8.0_77
            +1 javac 0m 23s the patch passed
            +1 compile 0m 22s the patch passed with JDK v1.7.0_95
            +1 javac 0m 22s the patch passed
            +1 checkstyle 0m 16s the patch passed
            +1 mvnsite 0m 27s the patch passed
            +1 mvneclipse 0m 15s the patch passed
            +1 whitespace 0m 0s Patch has no whitespace issues.
            +1 findbugs 0m 52s the patch passed
            +1 javadoc 0m 20s the patch passed with JDK v1.8.0_77
            +1 javadoc 0m 18s the patch passed with JDK v1.7.0_95
            +1 unit 10m 41s hadoop-distcp in the patch passed with JDK v1.8.0_77.
            +1 unit 9m 54s hadoop-distcp in the patch passed with JDK v1.7.0_95.
            +1 asflicense 0m 29s Patch does not generate ASF License warnings.
            39m 39s



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:fbe3e86
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch
            JIRA Issue HADOOP-13034
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 45af45862b20 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / d8b729e
            Default Java 1.7.0_95
            Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
            findbugs v3.0.0
            JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9110/testReport/
            modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
            Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9110/console
            Powered by Apache Yetus 0.2.0 http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 41s trunk passed +1 compile 0m 28s trunk passed with JDK v1.8.0_77 +1 compile 0m 25s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 0m 41s trunk passed +1 javadoc 0m 24s trunk passed with JDK v1.8.0_77 +1 javadoc 0m 22s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 24s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_77 +1 javac 0m 23s the patch passed +1 compile 0m 22s the patch passed with JDK v1.7.0_95 +1 javac 0m 22s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 27s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 52s the patch passed +1 javadoc 0m 20s the patch passed with JDK v1.8.0_77 +1 javadoc 0m 18s the patch passed with JDK v1.7.0_95 +1 unit 10m 41s hadoop-distcp in the patch passed with JDK v1.8.0_77. +1 unit 9m 54s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 29s Patch does not generate ASF License warnings. 39m 39s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch JIRA Issue HADOOP-13034 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 45af45862b20 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d8b729e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9110/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9110/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            templedf Daniel Templeton added a comment -

            LGTM. Looks like you got all the ones that were missing.

            Show
            templedf Daniel Templeton added a comment - LGTM. Looks like you got all the ones that were missing.
            Hide
            Permalink
            stevel@apache.org Steve Loughran added a comment -

            +1

            Show
            stevel@apache.org Steve Loughran added a comment - +1
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 11s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
            +1 mvninstall 6m 56s trunk passed
            +1 compile 0m 17s trunk passed
            +1 checkstyle 0m 14s trunk passed
            +1 mvnsite 0m 21s trunk passed
            +1 mvneclipse 0m 46s trunk passed
            +1 findbugs 0m 24s trunk passed
            +1 javadoc 0m 13s trunk passed
            +1 mvninstall 0m 15s the patch passed
            +1 compile 0m 14s the patch passed
            +1 javac 0m 14s the patch passed
            +1 checkstyle 0m 11s the patch passed
            +1 mvnsite 0m 18s the patch passed
            +1 mvneclipse 0m 10s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 0m 28s the patch passed
            +1 javadoc 0m 10s the patch passed
            +1 unit 9m 15s hadoop-distcp in the patch passed.
            +1 asflicense 0m 15s The patch does not generate ASF License warnings.
            21m 52s



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:85209cc
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch
            JIRA Issue HADOOP-13034
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 0c5a2574f2fc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 23c3ff8
            Default Java 1.8.0_91
            findbugs v3.0.0
            Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9886/testReport/
            modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
            Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9886/console
            Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 56s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 46s trunk passed +1 findbugs 0m 24s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 15s the patch passed +1 compile 0m 14s the patch passed +1 javac 0m 14s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 28s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 9m 15s hadoop-distcp in the patch passed. +1 asflicense 0m 15s The patch does not generate ASF License warnings. 21m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch JIRA Issue HADOOP-13034 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0c5a2574f2fc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 23c3ff8 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9886/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9886/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            aw Allen Wittenauer added a comment -

            +1 committed to trunk.

            Thanks!

            Show
            aw Allen Wittenauer added a comment - +1 committed to trunk. Thanks!
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Hadoop-trunk-Commit #10027 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10027/)
            HADOOP-13034. Log message about input options in distcp lacks some items (aw: rev 422c73a8657d8699920f7db13d4be200e16c4272)

            • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
            • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10027 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10027/ ) HADOOP-13034 . Log message about input options in distcp lacks some items (aw: rev 422c73a8657d8699920f7db13d4be200e16c4272) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 19s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
            +1 mvninstall 6m 37s trunk passed
            +1 compile 0m 17s trunk passed
            +1 checkstyle 0m 14s trunk passed
            +1 mvnsite 0m 21s trunk passed
            +1 mvneclipse 0m 16s trunk passed
            +1 findbugs 0m 30s trunk passed
            +1 javadoc 0m 12s trunk passed
            +1 mvninstall 0m 16s the patch passed
            +1 compile 0m 15s the patch passed
            +1 javac 0m 15s the patch passed
            +1 checkstyle 0m 11s the patch passed
            +1 mvnsite 0m 18s the patch passed
            +1 mvneclipse 0m 10s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 0m 29s the patch passed
            +1 javadoc 0m 10s the patch passed
            +1 unit 9m 32s hadoop-distcp in the patch passed.
            +1 asflicense 0m 14s The patch does not generate ASF License warnings.
            21m 38s



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:85209cc
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch
            JIRA Issue HADOOP-13034
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 0710bfa05f9b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / 23c3ff8
            Default Java 1.8.0_91
            findbugs v3.0.0
            Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9892/testReport/
            modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
            Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9892/console
            Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 37s trunk passed +1 compile 0m 17s trunk passed +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 21s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 12s trunk passed +1 mvninstall 0m 16s the patch passed +1 compile 0m 15s the patch passed +1 javac 0m 15s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 18s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 29s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 9m 32s hadoop-distcp in the patch passed. +1 asflicense 0m 14s The patch does not generate ASF License warnings. 21m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:85209cc JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799247/HADOOP-13034.1.patch JIRA Issue HADOOP-13034 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0710bfa05f9b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 23c3ff8 Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9892/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9892/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            bwtakacy Takashi Ohnishi added a comment -

            Thank you Allen Wittenauer for committing!
            Thank you Daniel Templeton and Steve Loughran for reveiwing!!

            Show
            bwtakacy Takashi Ohnishi added a comment - Thank you Allen Wittenauer for committing! Thank you Daniel Templeton and Steve Loughran for reveiwing!!
            Hide
            Permalink
            yzhangal Yongjun Zhang added a comment -

            Thanks you guys for the earlier work, I cherry-picked to branch-2.

            Show
            yzhangal Yongjun Zhang added a comment - Thanks you guys for the earlier work, I cherry-picked to branch-2.

              People

              • Assignee:
                bwtakacy Takashi Ohnishi
                Reporter:
                bwtakacy Takashi Ohnishi
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development