Hadoop Common
  1. Hadoop Common
  2. HADOOP-8431

Running distcp wo args throws IllegalArgumentException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Running distcp w/o args results in the following:

      hadoop-3.0.0-SNAPSHOT $ ./bin/hadoop distcp
      12/05/23 18:49:04 ERROR tools.DistCp: Invalid arguments: 
      java.lang.IllegalArgumentException: Target path not specified
      	at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86)
      	at org.apache.hadoop.tools.DistCp.run(DistCp.java:102)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.hadoop.tools.DistCp.main(DistCp.java:368)
      Invalid arguments: Target path not specified
      
      1. diff1.txt
        0.9 kB
        Sandy Ryza

        Activity

        Hide
        Steve Loughran added a comment -

        What was the expected behaviour? A usage() statement?

        Show
        Steve Loughran added a comment - What was the expected behaviour? A usage() statement?
        Hide
        Daisuke Kobayashi added a comment -

        I guess this has been fixed by MAPREDUCE-2765. If args are null, OptionsParser.usage() method would be called in DistCp.java.

        Show
        Daisuke Kobayashi added a comment - I guess this has been fixed by MAPREDUCE-2765 . If args are null, OptionsParser.usage() method would be called in DistCp.java.
        Hide
        Daisuke Kobayashi added a comment -

        I'll close this issue.

        Show
        Daisuke Kobayashi added a comment - I'll close this issue.
        Hide
        Sho Shimauchi added a comment -

        closing this JIRA.
        I've checked with Daisuke this issue is already fixed in MAPREDUCE-2765.
        Thanks for checking!

        Show
        Sho Shimauchi added a comment - closing this JIRA. I've checked with Daisuke this issue is already fixed in MAPREDUCE-2765 . Thanks for checking!
        Hide
        Eli Collins added a comment -

        I just tried this and confirmed it still fails with the latest build. In the future please try to reproduce the issue before you close it.

        hadoop-3.0.0-SNAPSHOT $ ./bin/hadoop distcp
        12/07/23 19:21:48 ERROR tools.DistCp: Invalid arguments:
        java.lang.IllegalArgumentException: Target path not specified
        at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86)
        at org.apache.hadoop.tools.DistCp.run(DistCp.java:102)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.tools.DistCp.main(DistCp.java:368)

        Show
        Eli Collins added a comment - I just tried this and confirmed it still fails with the latest build. In the future please try to reproduce the issue before you close it. hadoop-3.0.0-SNAPSHOT $ ./bin/hadoop distcp 12/07/23 19:21:48 ERROR tools.DistCp: Invalid arguments: java.lang.IllegalArgumentException: Target path not specified at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86) at org.apache.hadoop.tools.DistCp.run(DistCp.java:102) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.tools.DistCp.main(DistCp.java:368)
        Hide
        Sandy Ryza added a comment -

        I reproduced it as well. A usage() statement is printed in addition to the error - is the change to be made just removing the "Invalid arguments: Target path not specified" printed to System.err? Or the ERROR log statement as well?

        Show
        Sandy Ryza added a comment - I reproduced it as well. A usage() statement is printed in addition to the error - is the change to be made just removing the "Invalid arguments: Target path not specified" printed to System.err? Or the ERROR log statement as well?
        Hide
        Eli Collins added a comment -

        It should just print the usage. Eg in the following I'd remove the ERROR log, the IllegalArgumentException backtrace and the "Invalid arguments" log.

        $ ./bin/hadoop distcp
        12/09/05 20:38:26 ERROR tools.DistCp: Invalid arguments: 
        java.lang.IllegalArgumentException: Target path not specified
        	at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86)
        	at org.apache.hadoop.tools.DistCp.run(DistCp.java:102)
        	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        	at org.apache.hadoop.tools.DistCp.main(DistCp.java:368)
        Invalid arguments: Target path not specified
        usage: distcp OPTIONS [source_path...] <target_path>
                      OPTIONS
         -async                 Should distcp execution be blocking
         -atomic                Commit all changes or none
         -bandwidth <arg>       Specify bandwidth per map in MB
         -delete                Delete from target, files missing in source
         -f <arg>               List of files that need to be copied
         -filelimit <arg>       (Deprecated!) Limit number of files copied to <= n
         -i                     Ignore failures during copy
         -log <arg>             Folder on DFS where distcp execution logs are
                                saved
         -m <arg>               Max number of concurrent maps to use for copy
         -mapredSslConf <arg>   Configuration for ssl config file, to use with
                                hftps://
         -overwrite             Choose to overwrite target files unconditionally,
                                even if they exist.
         -p <arg>               preserve status (rbugp)(replication, block-size,
                                user, group, permission)
         -sizelimit <arg>       (Deprecated!) Limit number of files copied to <= n
                                bytes
         -skipcrccheck          Whether to skip CRC checks between source and
                                target paths.
         -strategy <arg>        Copy strategy to use. Default is dividing work
                                based on file sizes
         -tmp <arg>             Intermediate work path to be used for atomic
                                commit
         -update                Update target, copying only missingfiles or
                                directories
        
        Show
        Eli Collins added a comment - It should just print the usage. Eg in the following I'd remove the ERROR log, the IllegalArgumentException backtrace and the "Invalid arguments" log. $ ./bin/hadoop distcp 12/09/05 20:38:26 ERROR tools.DistCp: Invalid arguments: java.lang.IllegalArgumentException: Target path not specified at org.apache.hadoop.tools.OptionsParser.parse(OptionsParser.java:86) at org.apache.hadoop.tools.DistCp.run(DistCp.java:102) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.tools.DistCp.main(DistCp.java:368) Invalid arguments: Target path not specified usage: distcp OPTIONS [source_path...] <target_path> OPTIONS -async Should distcp execution be blocking -atomic Commit all changes or none -bandwidth <arg> Specify bandwidth per map in MB -delete Delete from target, files missing in source -f <arg> List of files that need to be copied -filelimit <arg> (Deprecated!) Limit number of files copied to <= n -i Ignore failures during copy -log <arg> Folder on DFS where distcp execution logs are saved -m <arg> Max number of concurrent maps to use for copy -mapredSslConf <arg> Configuration for ssl config file, to use with hftps:// -overwrite Choose to overwrite target files unconditionally, even if they exist. -p <arg> preserve status (rbugp)(replication, block-size, user, group, permission) -sizelimit <arg> (Deprecated!) Limit number of files copied to <= n bytes -skipcrccheck Whether to skip CRC checks between source and target paths. -strategy <arg> Copy strategy to use. Default is dividing work based on file sizes -tmp <arg> Intermediate work path to be used for atomic commit -update Update target, copying only missingfiles or directories
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12544091/diff1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-distcp.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1415//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1415//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12544091/diff1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-distcp. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1415//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1415//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        +1 looks great, verified the fix on a trunk tarball.

        Show
        Eli Collins added a comment - +1 looks great, verified the fix on a trunk tarball.
        Hide
        Eli Collins added a comment -

        I've committed this and merged to branch-2. Thanks Sandy!

        Show
        Eli Collins added a comment - I've committed this and merged to branch-2. Thanks Sandy!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2764 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2764/)
        HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2764 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2764/ ) HADOOP-8431 . Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2701 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2701/)
        HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2701 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2701/ ) HADOOP-8431 . Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2725 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2725/)
        HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2725 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2725/ ) HADOOP-8431 . Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Hide
        Sho Shimauchi added a comment -

        Hi Eli and Sandy,

        I apologize for not having checked this issue carefully.
        Thanks for taking this and writing a patch!

        Show
        Sho Shimauchi added a comment - Hi Eli and Sandy, I apologize for not having checked this issue carefully. Thanks for taking this and writing a patch!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/)
        HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1158/ ) HADOOP-8431 . Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/)
        HADOOP-8431. Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1189 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1189/ ) HADOOP-8431 . Running distcp wo args throws IllegalArgumentException. Contributed by Sandy Ryza (Revision 1381840) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381840 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java

          People

          • Assignee:
            Sandy Ryza
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development