hadoop dfs -put should support multiple src

• Key: HADOOP-3091
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Lohit Vijayarenu
• Reporter: Lohit Vijayarenu
• Affects Version/s: 0.16.2
• Fix Version/s: 0.17.0

Description

It would be good to have hadoop dfs -put to support multiple sources like LINUX cp
hadoop dfs -put <localsrc> ... <dst>

Comments

[Lohit Vijayarenu]

Here is a patch which takes care of this for -put/-copyFromLocal and -moveFromLocal

[Hadoop QA]

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378608/HADOOP-3091-1.patch
against trunk revision 619744.

@author +1. The patch does not contain any @author tags.

tests included +1. The patch appears to include 5 new or modified tests.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2059/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2059/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2059/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2059/console

This message is automatically generated.

[Mahadev konar]

Path dstPath = new Path(dstf);
+ FileSystem dstFs = dstPath.getFileSystem(getConf());
+ dstFs.copyFromLocalFile(false, false, srcs, new Path(dstf));
+ }

• creates path twice for the same string.
• the automatic directory creation should not be there. We should error out if the directory does not exist.
• also the hdfs_shell documentation (forrest) needs to be changed

[Lohit Vijayarenu]

Thanks Mahadev. Attaching new patch modifying as you suggested.

[Mahadev konar]

+1

[Hadoop QA]

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378657/HADOOP-3091-2.patch
against trunk revision 619744.

@author +1. The patch does not contain any @author tags.

tests included +1. The patch appears to include 5 new or modified tests.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2069/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2069/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2069/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2069/console

This message is automatically generated.

[Chris Douglas]

I just committed this. Thanks, Lohit!

[Hudson]

Integrated in Hadoop-trunk #443 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/443/)