Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.203.0, 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: distcp, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      DistCpV2 added to hadoop-tools.
    • Tags:
      distcp distcpv2 DynamicInputFormat

      Description

      This is a slightly modified version of the DistCp rewrite that Yahoo uses in production today. The rewrite was ground-up, with specific focus on:
      1. improved startup time (postponing as much work as possible to the MR job)
      2. support for multiple copy-strategies
      3. new features (e.g. -atomic, -async, -bandwidth.)
      4. improved programmatic use
      Some effort has gone into refactoring what used to be achieved by a single large (1.7 KLOC) source file, into a design that (hopefully) reads better too.

      The proposed DistCpV2 preserves command-line-compatibility with the old version, and should be a drop-in replacement.

      New to v2:

      1. Copy-strategies and the DynamicInputFormat:
      A copy-strategy determines the policy by which source-file-paths are distributed between map-tasks. (These boil down to the choice of the input-format.)
      If no strategy is explicitly specified on the command-line, the policy chosen is "uniform size", where v2 behaves identically to old-DistCp. (The number of bytes transferred by each map-task is roughly equal, at a per-file granularity.)
      Alternatively, v2 ships with a "dynamic" copy-strategy (in the DynamicInputFormat). This policy acknowledges that
      (a) dividing files based only on file-size might not be an even distribution (E.g. if some datanodes are slower than others, or if some files are skipped.)
      (b) a "static" association of a source-path to a map increases the likelihood of long-tails during copy.
      The "dynamic" strategy divides the list-of-source-paths into a number (> nMaps) of smaller parts. When each map completes its current list of paths, it picks up a new list to process, if available. So if a map-task is stuck on a slow (and not necessarily large) file, other maps can pick up the slack. The thinner the file-list is sliced, the greater the parallelism (and the lower the chances of long-tails). Within reason, of course: the number of these short-lived list-files is capped at an overridable maximum.
      Internal benchmarks against source/target clusters with some slow(ish) datanodes have indicated significant performance gains when using the dynamic-strategy. Gains are most pronounced when nFiles greatly exceeds nMaps.
      Please note that the DynamicInputFormat might prove useful outside of DistCp. It is hence available as a mapred/lib, unfettered to DistCpV2. Also note that the copy-strategies have no bearing on the CopyMapper.map() implementation.

      2. Improved startup-time and programmatic use:
      When the old-DistCp runs with -update, and creates the list-of-source-paths, it attempts to filter out files that might be skipped (by comparing file-sizes, checksums, etc.) This significantly increases the startup time (or the time spent in serial processing till the MR job is launched), blocking the calling-thread. This becomes pronounced as nFiles increases. (Internal benchmarks have seen situations where more time is spent setting up the job than on the actual transfer.)
      DistCpV2 postpones as much work as possible to the MR job. The file-listing isn't filtered until the map-task runs (at which time, identical files are skipped). DistCpV2 can now be run "asynchronously". The program quits at job-launch, logging the job-id for tracking. Programmatically, the DistCp.execute() returns a Job instance for progress-tracking.

      3. New features:
      (a) -async: As described in #2.
      (b) -atomic: Data is copied to a (user-specifiable) tmp-location, and then moved atomically to destination.
      (c) -bandwidth: Enforces a limit on the bandwidth consumed per map.
      (d) -strategy: As above.

      A more comprehensive description the newer features, how the dynamic-strategy works, etc. is available in src/site/xdoc/, and in the pdf that's generated therefrom, during the build.

      High on the list of things to do is support to parallelize copies on a per-block level. (i.e. Incorporation of HDFS-222.)

      I look forward to comments, suggestions and discussion that will hopefully ensue. I have this running against Hadoop 0.20.203.0. I also have a port to 0.23.0 (complete with unit-tests).

      P.S.
      A tip of the hat to Srikanth (Sundarrajan) and Venkatesh (Seetharamaiah), for ideas, code, reviews and guidance. Although much of the code is mine, the idea to use the DFS to implement "dynamic" input-splits wasn't.

      1. 2765_hadoop-branch-0.23.patch
        397 kB
        Mithun Radhakrishnan
      2. 2765_trunk.patch
        397 kB
        Mithun Radhakrishnan
      3. distcpv2_hadoop-0.23.1.patch
        396 kB
        Mithun Radhakrishnan
      4. distcpv2_hadoop-0.23.1.patch
        396 kB
        Mithun Radhakrishnan
      5. distcpv2_hadoop-trunk.patch
        396 kB
        Mithun Radhakrishnan
      6. distcpv2_patch_0.23.1-SNAPSHOT_tucu_reviewed.patch
        397 kB
        Mithun Radhakrishnan
      7. distcpv2_patch_hadoop-trunk_tucu_reviewed.patch
        397 kB
        Mithun Radhakrishnan
      8. distcpv2_trunk_post_review_1.patch
        403 kB
        Mithun Radhakrishnan
      9. distcpv2_trunk.patch
        402 kB
        Mithun Radhakrishnan
      10. distcpv2.20.203.patch
        419 kB
        Mithun Radhakrishnan

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          8d 13h 40m 2 Mithun Radhakrishnan 06/Jan/12 21:07
          In Progress In Progress Open Open
          160d 22h 47m 2 Mithun Radhakrishnan 19/Jan/12 00:32
          Open Open Patch Available Patch Available
          2h 24m 1 Mithun Radhakrishnan 19/Jan/12 02:56
          Patch Available Patch Available Resolved Resolved
          7d 3h 53m 1 Mahadev konar 26/Jan/12 06:50
          Resolved Resolved Closed Closed
          38d 19h 59m 1 Arun C Murthy 05/Mar/12 02:49
          Hide
          Venkatesh Seetharam added a comment -

          Tianying Chang, MAPREDUCE-5081 backported this to hadoop-1.

          Show
          Venkatesh Seetharam added a comment - Tianying Chang , MAPREDUCE-5081 backported this to hadoop-1.
          Hide
          Tianying Chang added a comment -

          Mithun Radhakrishnan Is it possible to use your distcp v2 in Mapreduce 1? It seems has some compatibility issue for getStagingDir(). Is there a way to make it working without have to install YARN.?

          Show
          Tianying Chang added a comment - Mithun Radhakrishnan Is it possible to use your distcp v2 in Mapreduce 1? It seems has some compatibility issue for getStagingDir(). Is there a way to make it working without have to install YARN.?
          Hide
          Mithun Radhakrishnan added a comment -

          Hello, Steven.

          It's been a while, but I think the idea was that most settings could be put in the Configuration used to create DistCp.

          But yes, I definitely see value in changing createJob() to a protected method. I'm curious to see in what direction you're considering extending DistCp.

          Show
          Mithun Radhakrishnan added a comment - Hello, Steven. It's been a while, but I think the idea was that most settings could be put in the Configuration used to create DistCp. But yes, I definitely see value in changing createJob() to a protected method. I'm curious to see in what direction you're considering extending DistCp.
          Hide
          Steven Magana-Zook added a comment -

          Why, in the org.apache.hadoop.tools.DistCp class, is the createJob method private instead of protected? In the programmatic way of using the DistCp class, I can see use cases where derived classes would like to modify the job settings (add parameters to the configuration for use in the mappers, extend distcp by adding reducers, etc.).

          Is it because there is a different way of accomplishing these extensions in derived classes without having to overload the createJob method?

          I see in the DistCpV2 project that the createJob method is in fact protected, but that does not seem to have made it into the official Hadoop DistCp tool as part of the rewrite.

          Show
          Steven Magana-Zook added a comment - Why, in the org.apache.hadoop.tools.DistCp class, is the createJob method private instead of protected? In the programmatic way of using the DistCp class, I can see use cases where derived classes would like to modify the job settings (add parameters to the configuration for use in the mappers, extend distcp by adding reducers, etc.). Is it because there is a different way of accomplishing these extensions in derived classes without having to overload the createJob method? I see in the DistCpV2 project that the createJob method is in fact protected, but that does not seem to have made it into the official Hadoop DistCp tool as part of the rewrite.
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to MAPREDUCE-5081 [ MAPREDUCE-5081 ]
          Hide
          Eli Collins added a comment -

          Why did this change add a test class that doesn't run? Did TestDistCp ever pass with this change?

          +@Ignore
          +public class TestDistCp {
          

          Filed HADOOP-8768 for this, doesn't seem like distcp has any test coverage.

          Show
          Eli Collins added a comment - Why did this change add a test class that doesn't run? Did TestDistCp ever pass with this change? +@Ignore + public class TestDistCp { Filed HADOOP-8768 for this, doesn't seem like distcp has any test coverage.
          Eli Collins made changes -
          Link This issue relates to HADOOP-8768 [ HADOOP-8768 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #970 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/970/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048
          Files :

          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #970 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/970/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048 Files : /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/trunk/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/172/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #172 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/172/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #150 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/150/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #150 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/150/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #937 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/937/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048
          Files :

          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #937 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/937/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048 Files : /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/trunk/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1606 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1606/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048
          Files :

          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1606 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1606/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048 Files : /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #438 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/438/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #438 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/438/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1589 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1589/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048
          Files :

          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1589 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1589/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048 Files : /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #413 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/413/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #413 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/413/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1605 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1605/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1605 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1605/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/trunk/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #422 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/422/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #422 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/422/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236049 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-project/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/README /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/branches/branch-0.23/hadoop-tools/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1661 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1661/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048
          Files :

          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1661 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1661/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) - wrong version in poms MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236048 Files : /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/trunk/hadoop-tools/pom.xml
          Mahadev konar made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Hide
          Mahadev konar added a comment -

          I just committed this. Thanks Mithun!

          Show
          Mahadev konar added a comment - I just committed this. Thanks Mithun!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1588 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1588/)
          MAPREDUCE-2765. DistCp Rewrite. (Mithun Radhakrishnan via mahadev)

          mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources
          • /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml
          • /hadoop/common/trunk/hadoop-tools/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1588 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1588/ ) MAPREDUCE-2765 . DistCp Rewrite. (Mithun Radhakrishnan via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1236045 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp /hadoop/common/trunk/hadoop-tools/hadoop-distcp/README /hadoop/common/trunk/hadoop-tools/hadoop-distcp/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/CopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpConstants.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/FileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/GlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/SimpleCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableDirectoryCreateCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/UniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputChunk.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/lib/DynamicRecordReader.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/RetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/resources/distcp-default.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/fml/faq.fml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/pdf.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/appendix.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/architecture.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/cli.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/index.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/site/xdoc/usage.xml /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/StubContext.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCp.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestFileBasedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestGlobbedCopyListing.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyOutputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestUniformSizeInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/lib/TestDynamicInputFormat.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestDistCpUtils.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestRetriableCommand.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/util/TestThrottledInputStream.java /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/resources/sslConfig.xml /hadoop/common/trunk/hadoop-tools/pom.xml
          Hide
          Mahadev konar added a comment -

          Looks like when I reran the command, it started working all fine. I will go ahead and commit this.

          Show
          Mahadev konar added a comment - Looks like when I reran the command, it started working all fine. I will go ahead and commit this.
          Hide
          Mahadev konar added a comment -

          @Mithun,
          Any reason the cleaning up of m2 repo fixes it? No other way around it? This will cause all builds to break if we check this is as it is.

          Show
          Mahadev konar added a comment - @Mithun, Any reason the cleaning up of m2 repo fixes it? No other way around it? This will cause all builds to break if we check this is as it is.
          Hide
          Mithun Radhakrishnan added a comment -

          Clearing(/moving) the .m2/repository directory (once) should do the trick, from what I gather from John. As far as I can tell, this is all that should need doing.

          Show
          Mithun Radhakrishnan added a comment - Clearing(/moving) the .m2/repository directory (once) should do the trick, from what I gather from John. As far as I can tell, this is all that should need doing.
          Hide
          Mahadev konar added a comment -

          @Mithun,
          I am not sure what you mean by:

          My guess would be that the pdf-plugin had issues with the fml-xsd, that have since been resolved.

          I am still seeing the issue. Something I need to do to make it go away?

          Show
          Mahadev konar added a comment - @Mithun, I am not sure what you mean by: My guess would be that the pdf-plugin had issues with the fml-xsd, that have since been resolved. I am still seeing the issue. Something I need to do to make it go away?
          Hide
          Mithun Radhakrishnan added a comment -

          To clarify, since one doesn't need to clear .m2/ for every build (i.e. since this is one-off), can this be submitted now? I'm loath to delay this longer.

          Show
          Mithun Radhakrishnan added a comment - To clarify, since one doesn't need to clear .m2/ for every build (i.e. since this is one-off), can this be submitted now? I'm loath to delay this longer.
          Hide
          Mithun Radhakrishnan added a comment -

          @John: Thanks for trying that out.

          @Mahadev: I'm not sure what was in the .m2/ cache before, but I've not had to clean the cache to build this. :] The quoted error seems to point at http://maven.apache.org/xsd/fml-1.0.1.xsd::2771. My guess would be that the pdf-plugin had issues with the fml-xsd, that have since been resolved.

          Show
          Mithun Radhakrishnan added a comment - @John: Thanks for trying that out. @Mahadev: I'm not sure what was in the .m2/ cache before, but I've not had to clean the cache to build this. :] The quoted error seems to point at http://maven.apache.org/xsd/fml-1.0.1.xsd::2771 . My guess would be that the pdf-plugin had issues with the fml-xsd, that have since been resolved.
          Hide
          Mahadev konar added a comment -

          @John,
          Thats an issue. There is definitely something wrong if we need to clean our m2 caches for this.

          Show
          Mahadev konar added a comment - @John, Thats an issue. There is definitely something wrong if we need to clean our m2 caches for this.
          Hide
          John George added a comment -

          @Mahadev - I had the same error, but the build succeeded once I cleaned my ivy and m2 caches.

          Show
          John George added a comment - @Mahadev - I had the same error, but the build succeeded once I cleaned my ivy and m2 caches.
          Hide
          Mithun Radhakrishnan added a comment -

          How odd. I checked that the patch applied to branch-0.23 (http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23/) yesterday. I'll check again.

          Show
          Mithun Radhakrishnan added a comment - How odd. I checked that the patch applied to branch-0.23 ( http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23/ ) yesterday. I'll check again.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12511080/2765_hadoop-branch-0.23.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 66 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1634//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12511080/2765_hadoop-branch-0.23.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 66 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1634//console This message is automatically generated.
          Hide
          Mahadev konar added a comment -

          @Mithun,

          I am getting the following error when running:

          mvn clean package -DskipTests -Pdist -Dtar -Dmaven.javadoc.skip=true
          
          
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-pdf-plugin:1.1:pdf (pdf) on project hadoop-distcp: Error during document generation: Error parsing /Users/mahadev/workspace/hadoop-workspace/hadoop-git-0.23/hadoop-tools/hadoop-distcp/target/pdf/site.tmp/fml/faq.fml: Error validating the model: Fatal error:
          [ERROR] Public ID: null
          [ERROR] System ID: http://maven.apache.org/xsd/fml-1.0.1.xsd
          [ERROR] Line number: 2771
          [ERROR] Column number: 38
          [ERROR] Message: XML document structures must start and end within the same entity.
          [ERROR] -> [Help 1]
          [ERROR] 
          [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
          [ERROR] Re-run Maven using the -X switch to enable full debug logging.
          [ERROR] 
          [ERROR] For more information about the errors and possible solutions, please read the following articles:
          [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
          [ERROR] 
          [ERROR] After correcting the problems, you can resume the build with the command
          [ERROR]   mvn <goals> -rf :hadoop-distcp
          
          

          Can you please take a look at it?

          Show
          Mahadev konar added a comment - @Mithun, I am getting the following error when running: mvn clean package -DskipTests -Pdist -Dtar -Dmaven.javadoc.skip= true [ERROR] Failed to execute goal org.apache.maven.plugins:maven-pdf-plugin:1.1:pdf (pdf) on project hadoop-distcp: Error during document generation: Error parsing /Users/mahadev/workspace/hadoop-workspace/hadoop-git-0.23/hadoop-tools/hadoop-distcp/target/pdf/site.tmp/fml/faq.fml: Error validating the model: Fatal error: [ERROR] Public ID: null [ERROR] System ID: http: //maven.apache.org/xsd/fml-1.0.1.xsd [ERROR] Line number: 2771 [ERROR] Column number: 38 [ERROR] Message: XML document structures must start and end within the same entity. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch . [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http: //cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hadoop-distcp Can you please take a look at it?
          Hide
          Mithun Radhakrishnan added a comment -

          @Mahadev: I'm with you on that. I've updated the DistCpV2 patches and hit submit. (2765*patch). Thanks for reviewing.

          Show
          Mithun Radhakrishnan added a comment - @Mahadev: I'm with you on that. I've updated the DistCpV2 patches and hit submit. (2765*patch). Thanks for reviewing.
          Mithun Radhakrishnan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note DistCpV2 added to hadoop-tools.
          Mithun Radhakrishnan made changes -
          Attachment 2765_trunk.patch [ 12511079 ]
          Attachment 2765_hadoop-branch-0.23.patch [ 12511080 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Updated patches (yet again) for changes in trunk and branch/.

          Show
          Mithun Radhakrishnan added a comment - Updated patches (yet again) for changes in trunk and branch/.
          Mithun Radhakrishnan made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Mahadev konar made changes -
          Fix Version/s 0.23.1 [ 12318883 ]
          Affects Version/s 0.23.0 [ 12315570 ]
          Hide
          Mahadev konar added a comment -

          @Mithun,
          Are you going to make this jira patch available? I think we should commit this patch soon!

          Show
          Mahadev konar added a comment - @Mithun, Are you going to make this jira patch available? I think we should commit this patch soon!
          Mithun Radhakrishnan made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Mithun Radhakrishnan made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Amareshwari, would it be possible for this to be committed?

          Show
          Mithun Radhakrishnan added a comment - Amareshwari, would it be possible for this to be committed?
          Harsh J made changes -
          Link This issue blocks OOZIE-611 [ OOZIE-611 ]
          Arun C Murthy made changes -
          Priority Critical [ 2 ] Major [ 3 ]
          Mithun Radhakrishnan made changes -
          Hide
          Mithun Radhakrishnan added a comment -

          New patches uploaded, incorporating all of Tucu's comments, except for using an assembly. I guess that'll have to wait until we have a hadoop-tools-dist. (I'm also guessing that'll take time to get right.)

          (P.S. Thanks, Tucu. :])

          Show
          Mithun Radhakrishnan added a comment - New patches uploaded, incorporating all of Tucu's comments, except for using an assembly. I guess that'll have to wait until we have a hadoop-tools-dist. (I'm also guessing that'll take time to get right.) (P.S. Thanks, Tucu. :])
          Hide
          Alejandro Abdelnur added a comment -

          Feedback on the build part:

          • dir name should be hadoop-distcp instead of distpcv2

          pom.xml:

          • line 127+: why do you need to do this? This should be taken care in the same way as for hadoop-streaming MAPREDUCE-3435 (maybe this JIRA should be generalized to tools)
          • line 150+: the resources plugin version should be defined in the dependency management section of the hadoop-project/pom.xml
          • line 164: why do we need to generate the test JAR?
          • line 177+: not needed, this comes for parent pom.
          • line 214+: plugin management should be done in the hadoop-project/pom.xml

          documentation:

          • In different parts we are starting to use APT (worth considering)

          formatting:

          • in many files there are trailing spaces
          Show
          Alejandro Abdelnur added a comment - Feedback on the build part: dir name should be hadoop-distcp instead of distpcv2 pom.xml: line 127+: why do you need to do this? This should be taken care in the same way as for hadoop-streaming MAPREDUCE-3435 (maybe this JIRA should be generalized to tools) line 150+: the resources plugin version should be defined in the dependency management section of the hadoop-project/pom.xml line 164: why do we need to generate the test JAR? line 177+: not needed, this comes for parent pom. line 214+: plugin management should be done in the hadoop-project/pom.xml documentation: In different parts we are starting to use APT (worth considering) formatting: in many files there are trailing spaces
          Hide
          Mithun Radhakrishnan added a comment -

          Sorry I missed this before, but it looks like this bug has been made a blocker for 6448. The intention was for the new DistCp to replace the original one, but not in the immediate time-frame. (It'll take time to port this not to use FileSystem.) Which is also why old-distcp was left untouched in this patch.

          I'd appreciate comment/advice on whether this patch fits the bill, otherwise.

          Show
          Mithun Radhakrishnan added a comment - Sorry I missed this before, but it looks like this bug has been made a blocker for 6448. The intention was for the new DistCp to replace the original one, but not in the immediate time-frame. (It'll take time to port this not to use FileSystem.) Which is also why old-distcp was left untouched in this patch. I'd appreciate comment/advice on whether this patch fits the bill, otherwise.
          Mahadev konar made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          Component/s mrv2 [ 12314301 ]
          Hide
          Mithun Radhakrishnan added a comment -

          The updated patches are up.

          Thanks.

          Show
          Mithun Radhakrishnan added a comment - The updated patches are up. Thanks.
          Mithun Radhakrishnan made changes -
          Attachment distcpv2_hadoop-0.23.1.patch [ 12505590 ]
          Attachment distcpv2_hadoop-trunk.patch [ 12505591 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Yes, Mahadev. Apologies, I'll do that in just a moment. I'll also upload the patch that applies against trunk/.

          Show
          Mithun Radhakrishnan added a comment - Yes, Mahadev. Apologies, I'll do that in just a moment. I'll also upload the patch that applies against trunk/.
          Hide
          Mahadev konar added a comment -

          @Mithun,
          Can you please re upload the patch and also make sure you grant license to ASF for inclusion?

          Thanks

          Show
          Mahadev konar added a comment - @Mithun, Can you please re upload the patch and also make sure you grant license to ASF for inclusion? Thanks
          Mithun Radhakrishnan made changes -
          Attachment distcpv2_hadoop-0.23.1.patch [ 12505584 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Patch that applies against 0.23.1-SNAPSHOT.

          Show
          Mithun Radhakrishnan added a comment - Patch that applies against 0.23.1-SNAPSHOT.
          Hide
          Mithun Radhakrishnan added a comment -

          Working on (only) this right now. Some of the tests are broken again. Should have an updated patch shortly.

          Show
          Mithun Radhakrishnan added a comment - Working on (only) this right now. Some of the tests are broken again. Should have an updated patch shortly.
          Hide
          Alejandro Abdelnur added a comment -

          Hi Mithun, Any update on this? Thanks.

          Show
          Alejandro Abdelnur added a comment - Hi Mithun, Any update on this? Thanks.
          Hide
          Mithun Radhakrishnan added a comment -

          Not yet, Alejandro. I'll fix the patch up early next week. (Sorry, been occupied.)

          Show
          Mithun Radhakrishnan added a comment - Not yet, Alejandro. I'll fix the patch up early next week. (Sorry, been occupied.)
          Hide
          Alejandro Abdelnur added a comment -

          Any update on this JIRA?

          The module hadoop-tools is now available.

          Show
          Alejandro Abdelnur added a comment - Any update on this JIRA? The module hadoop-tools is now available.
          Harsh J made changes -
          Link This issue blocks HADOOP-6448 [ HADOOP-6448 ]
          Hide
          Alejandro Abdelnur added a comment -

          Comments on the POM:

          • 1 It should use hadoop-project POM as parent
          • 2 Version should be hardcoded, not property
          • 3 It should not define repositories (they are defined in hadoop-project POM)
          • 4 All dependencies should not have versions, thus taking the version from the hadoop-project POM dependenciesManagement section (if there is a new dependency introduced by distcp, it should be added to the hadoop-project POM dependenciesManagement section and in this POM without version as the rest, this ensure across all Hadoop modules the same versions of all JARs are used at compile/test/dist time)
          • 5 Why aspect JARs have compile scope (no scope means compile scope)?
          • 6 Similarly Plugin versions should be defined in the pluginManagement section in the hadoop-project POM
          • 7 For checkstyle/findbugs/clover see to reuse the settings of hadoop-project POM
          • 8 clover license is already defined in hadoop-project POM (and the one here seems an internal Y! location)
          • 9 The surefire plugin <systemProperties> section, you should use the syntax <PROP_NAME>PROP_VALUE</PROP_NAME> (see hadoop-project POM)
          • 10 It seems the dependency plugin is being used to create the tool dist in the target/ dir. This is not correct, an assembly should be used. Ideally the existing hadoop-dist assembly should be reused.

          #10 brings up to the table how do we want to run tools? How their classpath/librarypath is created, which JARs are used bundled with the tool, etc.

          Finally, based on the outcome of the discussion of where tools should be (trunk/hadoop-tools-project/ or at /trunk/hadoop-project/hadoop-tools) the patch should be updated to it.

          Show
          Alejandro Abdelnur added a comment - Comments on the POM: 1 It should use hadoop-project POM as parent 2 Version should be hardcoded, not property 3 It should not define repositories (they are defined in hadoop-project POM) 4 All dependencies should not have versions, thus taking the version from the hadoop-project POM dependenciesManagement section (if there is a new dependency introduced by distcp, it should be added to the hadoop-project POM dependenciesManagement section and in this POM without version as the rest, this ensure across all Hadoop modules the same versions of all JARs are used at compile/test/dist time) 5 Why aspect JARs have compile scope (no scope means compile scope)? 6 Similarly Plugin versions should be defined in the pluginManagement section in the hadoop-project POM 7 For checkstyle/findbugs/clover see to reuse the settings of hadoop-project POM 8 clover license is already defined in hadoop-project POM (and the one here seems an internal Y! location) 9 The surefire plugin <systemProperties> section, you should use the syntax <PROP_NAME>PROP_VALUE</PROP_NAME> (see hadoop-project POM) 10 It seems the dependency plugin is being used to create the tool dist in the target/ dir. This is not correct, an assembly should be used. Ideally the existing hadoop-dist assembly should be reused. #10 brings up to the table how do we want to run tools? How their classpath/librarypath is created, which JARs are used bundled with the tool, etc. Finally, based on the outcome of the discussion of where tools should be (trunk/hadoop-tools-project/ or at /trunk/hadoop- project/hadoop -tools) the patch should be updated to it.
          Hide
          Allen Wittenauer added a comment -

          IIRC, -filelimit and -sizelimit were the original attempts to try and control bandwidth used. They did get used for a while until it became evident that when doing an update they broke in horrid ways (as pointed out above).

          The one thing that sticks in mind is if they would still be useful in any use cases involving quotas. But I suppose that ultimately the lack of these options could be worked around by manipulating the command line arguments given to distcp, right?

          Show
          Allen Wittenauer added a comment - IIRC, -filelimit and -sizelimit were the original attempts to try and control bandwidth used. They did get used for a while until it became evident that when doing an update they broke in horrid ways (as pointed out above). The one thing that sticks in mind is if they would still be useful in any use cases involving quotas. But I suppose that ultimately the lack of these options could be worked around by manipulating the command line arguments given to distcp, right?
          Mithun Radhakrishnan made changes -
          Attachment distcpv2_trunk_post_review_1.patch [ 12491487 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Patch for latest trunk. Incorporates Amareshwari's review comments.

          Show
          Mithun Radhakrishnan added a comment - Patch for latest trunk. Incorporates Amareshwari's review comments.
          Hide
          Mithun Radhakrishnan added a comment -

          Thanks for the thorough review, Amareshwari. I've incorporated a majority of your suggestions, save a couple.

          1. Constants for mapred.local.dir, mapred.job.name, etc.: I've switched to using the constants from JobContext, as you've suggested.
          2. ssl.client.* conf-labels: I've moved those to DistCpConstants.
          3. CONF_LABEL_TOTAL_*: The reason why the values of these constants don't start with "distcp" is that CONF_LABEL_TOTAL_NUMBER_OF_RECORDS is used from within DynamicInputFormat (which will eventually be independent of distcp). I've kept the other one for uniformity.
          4. Javadocs: You're right. I had missed the DistCp driver class. Thank you for pointing that out. I've also resolved all the warnings from the generation of Javadocs for the project.
          5. Job-id hack: I have removed the hack. I'm using Job.getJobID().toString() instead.
          6. CopyCommitter documentation: I've corrected the documentation for preserveFileAttributesForDirectories() pertaining to the target-root-path (which was previously misleading). I've also elaborated on the workings of deleteMissing(), etc.
          7. FileSplits with zero-length: For the moment, I've set the FileSplit to be set up with a fixed non-zero size. I'd intended to initialize it with the number of paths in the FileSplit, but that would be intrusive. If required, I'll change that in a future patch.
          8. Deleting attempt tmp-files: This is a very good suggestion. Incorporating it would be slightly invasive at this time. I'd like very much to include this at a later date. The cleanup would be much quicker.

          Findbugs seems to indicate 2 "bugs" that are false positives:
          1. One points at a possible race condition between the cleanup-handler shutdown-hook (from DistCp.java). There's no problem here, given that the cleanup-state is protected via synch. I've verified this with Amareshwari.
          2. The other is a silly one, indicating that some exceptions are caught and ignored. (Dead store to local variable).

          I'd like to suppress these 2 warnings, if that's alright.

          I'll post a newer patch (for hadoop-mapreduce) very shortly.

          Show
          Mithun Radhakrishnan added a comment - Thanks for the thorough review, Amareshwari. I've incorporated a majority of your suggestions, save a couple. 1. Constants for mapred.local.dir, mapred.job.name, etc.: I've switched to using the constants from JobContext, as you've suggested. 2. ssl.client.* conf-labels: I've moved those to DistCpConstants. 3. CONF_LABEL_TOTAL_*: The reason why the values of these constants don't start with "distcp" is that CONF_LABEL_TOTAL_NUMBER_OF_RECORDS is used from within DynamicInputFormat (which will eventually be independent of distcp). I've kept the other one for uniformity. 4. Javadocs: You're right. I had missed the DistCp driver class. Thank you for pointing that out. I've also resolved all the warnings from the generation of Javadocs for the project. 5. Job-id hack: I have removed the hack. I'm using Job.getJobID().toString() instead. 6. CopyCommitter documentation: I've corrected the documentation for preserveFileAttributesForDirectories() pertaining to the target-root-path (which was previously misleading). I've also elaborated on the workings of deleteMissing(), etc. 7. FileSplits with zero-length: For the moment, I've set the FileSplit to be set up with a fixed non-zero size. I'd intended to initialize it with the number of paths in the FileSplit, but that would be intrusive. If required, I'll change that in a future patch. 8. Deleting attempt tmp-files: This is a very good suggestion. Incorporating it would be slightly invasive at this time. I'd like very much to include this at a later date. The cleanup would be much quicker. Findbugs seems to indicate 2 "bugs" that are false positives: 1. One points at a possible race condition between the cleanup-handler shutdown-hook (from DistCp.java). There's no problem here, given that the cleanup-state is protected via synch. I've verified this with Amareshwari. 2. The other is a silly one, indicating that some exceptions are caught and ignored. (Dead store to local variable). I'd like to suppress these 2 warnings, if that's alright. I'll post a newer patch (for hadoop-mapreduce) very shortly.
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch is very close. Some comments on code:
          CopyMapper:

          String workDir = conf.get("mapred.local.dir") + "/work";

          Please use the String constant defined for this config. That would ease any change in the config name, deprecation and etc.

          Do you want to add ssl.client* config names to DistCp constants?

          DistCpConstants:

          public static final String CONF_LABEL_NUM_MAPS = "mapred.map.tasks";

          Please use the string constant defined for this config. This is a deprecated config and soon be removed.

          public static final String CONF_LABEL_TOTAL_BYTES_TO_BE_COPIED = "mapred.total.bytes.expected";

          public static final String CONF_LABEL_TOTAL_NUMBER_OF_RECORDS = "mapred.number.of.records";

          Shall we change these names to start with distcp.?

          All public classes and public methods need javadoc

          Can you check this once again? For ex. DistCp.java does not have javadoc

          DistCp:

          String userChosenName = getConf().get("mapred.job.name");

          job.getConfiguration().set("mapred.map.tasks.speculative.execution", "false");

          Please use the String constant defined for this config. That would ease any change in the config name, deprecation and etc.

          A hack to get the DistCp job id. New Job API doesn't return the job id

          This is no more required. MAPREDUCE-118 is resolved now.

          CopyCommitter:

          //Skip the root folder, preserve the status after atomic commit is complete

          Do you want to change this comment to reflect why we are skipping root folder?

          Deleting attempt temp files happens in each attempt. Why are we doing delete again in Committer? Committer should just delete the work path.

          The methods deleteMissing(), preserveFileAttributes() etc need more doc.

          DynamicInputFormat creates FileSplits with zero length. Instead should it be created with the size of chunk as the size of the split.

          I don't see any change for the above from the previous patch. Are you planning to do them?

          Also, please run findbugs, javadoc and releaseaudit and make sure there are no warnings.

          Show
          Amareshwari Sriramadasu added a comment - Patch is very close. Some comments on code: CopyMapper: String workDir = conf.get("mapred.local.dir") + "/work"; Please use the String constant defined for this config. That would ease any change in the config name, deprecation and etc. Do you want to add ssl.client* config names to DistCp constants? DistCpConstants: public static final String CONF_LABEL_NUM_MAPS = "mapred.map.tasks"; Please use the string constant defined for this config. This is a deprecated config and soon be removed. public static final String CONF_LABEL_TOTAL_BYTES_TO_BE_COPIED = "mapred.total.bytes.expected"; public static final String CONF_LABEL_TOTAL_NUMBER_OF_RECORDS = "mapred.number.of.records"; Shall we change these names to start with distcp.? All public classes and public methods need javadoc Can you check this once again? For ex. DistCp.java does not have javadoc DistCp: String userChosenName = getConf().get("mapred.job.name"); job.getConfiguration().set("mapred.map.tasks.speculative.execution", "false"); Please use the String constant defined for this config. That would ease any change in the config name, deprecation and etc. A hack to get the DistCp job id. New Job API doesn't return the job id This is no more required. MAPREDUCE-118 is resolved now. CopyCommitter: //Skip the root folder, preserve the status after atomic commit is complete Do you want to change this comment to reflect why we are skipping root folder? Deleting attempt temp files happens in each attempt. Why are we doing delete again in Committer? Committer should just delete the work path. The methods deleteMissing(), preserveFileAttributes() etc need more doc. DynamicInputFormat creates FileSplits with zero length. Instead should it be created with the size of chunk as the size of the split. I don't see any change for the above from the previous patch. Are you planning to do them? Also, please run findbugs, javadoc and releaseaudit and make sure there are no warnings.
          Hide
          Amareshwari Sriramadasu added a comment -

          One thing I like about this patch is the documentation in site/. Everything is very well described. Good work Mithun!

          Please generate the patch relative to hadoop-mapreduce(which was mapreduce before). The files would start with src/
          Your pom.xml contains a url to Yahoo! wiki. Please remove. Also remove all the references to Yahoo! In the patch. Searching for yahoo in the patch would give more details.
          The pom.xml is hardcoded to use 0.23.0-SNAPSHOT. Since this is part the hadoop project now, it should be changed to the current hadoop version, no?

          I'm going through the code again. Will update if I have any comments.

          Show
          Amareshwari Sriramadasu added a comment - One thing I like about this patch is the documentation in site/. Everything is very well described. Good work Mithun! Please generate the patch relative to hadoop-mapreduce(which was mapreduce before). The files would start with src/ Your pom.xml contains a url to Yahoo! wiki. Please remove. Also remove all the references to Yahoo! In the patch. Searching for yahoo in the patch would give more details. The pom.xml is hardcoded to use 0.23.0-SNAPSHOT. Since this is part the hadoop project now, it should be changed to the current hadoop version, no? I'm going through the code again. Will update if I have any comments.
          Hide
          Mithun Radhakrishnan added a comment -

          No, Luke. Not at this time. What we have now is a port of the code that works on 0.20.203.

          Parallel block copy is at the top of the TODO list, though.

          Show
          Mithun Radhakrishnan added a comment - No, Luke. Not at this time. What we have now is a port of the code that works on 0.20.203. Parallel block copy is at the top of the TODO list, though.
          Hide
          Luke Lu added a comment -

          Does the trunk patch incorporate MAPREDUCE-2257 (parallel block copy)?

          Show
          Luke Lu added a comment - Does the trunk patch incorporate MAPREDUCE-2257 (parallel block copy)?
          Hide
          Koji Noguchi added a comment -

          Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2?

          +1. I think we (Yahoo) requested but ended up not using it at all.

          Just to be clear

          The file-listing isn't filtered until the map-task runs

          This used to be the case in old old distcp. We changed that when we added this -filelimit feature (that we never used).

          Show
          Koji Noguchi added a comment - Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2? +1. I think we (Yahoo) requested but ended up not using it at all. Just to be clear The file-listing isn't filtered until the map-task runs This used to be the case in old old distcp. We changed that when we added this -filelimit feature (that we never used).
          Hide
          Mithun Radhakrishnan added a comment -

          I've now attached a patch that applies to hadoop-trunk. This takes care of nearly all the review comments so far, save the following:
          1. Preserve-status on the root-directory: I've left it out, because this can't really be handled. (For instance, if there are more than one source-path, it is ambiguous to determine the target-root's permissions.)
          2. The count in deleteMissing(): It turns out that this is accurate.

          I've also modified -atomic not to blow away the target-path, if it exists. This will need removing before an atomic commit can be made. As per Amareshwari's advice, this is checked before the copy is initiated. (There's also a check just before the commit is done, as a corner case. The copied data is retained in the temporary workspace.) Tests have been altered accordingly.

          (We briefly toyed with the idea of using -overwrite along with -atomic, to forcibly overwrite the target, but this mixes disjoint concerns, while only offering the dubious convenience of deleting pre-existing targets. We're leaving -atomic and -overwrite mutually exclusive.)

          I'm leaving it mavenized, so that it may be built from mapreduce/src/contrib/distcpv2/ by "mvn package". The jar may then be used with "bin/hadoop jar hadoop-distcp.jar [OPTIONS]". The usage is described at greater length in the pdf generated in the build.

          Thank you, reviewers.

          Show
          Mithun Radhakrishnan added a comment - I've now attached a patch that applies to hadoop-trunk. This takes care of nearly all the review comments so far, save the following: 1. Preserve-status on the root-directory: I've left it out, because this can't really be handled. (For instance, if there are more than one source-path, it is ambiguous to determine the target-root's permissions.) 2. The count in deleteMissing(): It turns out that this is accurate. I've also modified -atomic not to blow away the target-path, if it exists. This will need removing before an atomic commit can be made. As per Amareshwari's advice, this is checked before the copy is initiated. (There's also a check just before the commit is done, as a corner case. The copied data is retained in the temporary workspace.) Tests have been altered accordingly. (We briefly toyed with the idea of using -overwrite along with -atomic, to forcibly overwrite the target, but this mixes disjoint concerns, while only offering the dubious convenience of deleting pre-existing targets. We're leaving -atomic and -overwrite mutually exclusive.) I'm leaving it mavenized, so that it may be built from mapreduce/src/contrib/distcpv2/ by "mvn package". The jar may then be used with "bin/hadoop jar hadoop-distcp.jar [OPTIONS] ". The usage is described at greater length in the pdf generated in the build. Thank you, reviewers.
          Mithun Radhakrishnan made changes -
          Attachment distcpv2_trunk.patch [ 12490238 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Patch for hadoop-trunk.

          Show
          Mithun Radhakrishnan added a comment - Patch for hadoop-trunk.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2?

          Haven't looked at the code yet, but can clearly understand how useless these knobs are. +1 for not supporting them at all, we should probably continue to parse these options but just print a warning saying they are not supportable.

          Show
          Vinod Kumar Vavilapalli added a comment - Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2? Haven't looked at the code yet, but can clearly understand how useless these knobs are. +1 for not supporting them at all, we should probably continue to parse these options but just print a warning saying they are not supportable.
          Mithun Radhakrishnan made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Mithun Radhakrishnan added a comment -

          Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2?

          Old DistCp would use this to limit the number of files/bytes moved, but the results are a tad random, particularly in the -update case. (The first "n" files are added to the copy-list.) It sounds like a way to avoid a mis-configuration. I'm inclined to think it's not used often.

          Since DistCpV2 determines whether to skip files only in the Map-job, these would be pretty hard to implement.

          Show
          Mithun Radhakrishnan added a comment - Would the reviewers/watchers kindly comment on whether it's alright to deprecate the "-filelimit" and "-sizelimit" options, in DistCpV2? Old DistCp would use this to limit the number of files/bytes moved, but the results are a tad random, particularly in the -update case. (The first "n" files are added to the copy-list.) It sounds like a way to avoid a mis-configuration. I'm inclined to think it's not used often. Since DistCpV2 determines whether to skip files only in the Map-job, these would be pretty hard to implement.
          Hide
          Mithun Radhakrishnan added a comment -

          A couple of notes on the comments:

          0. In its current state, DistCpV2 is maven-ized. I'm considering not switching to ant and then switching back.

          1. CopyMapper behaviour: The quoted check is for overwriting a file when the source-path and the target-path are both complete paths to pre-existing files.
          "distcp /source/file.txt /target/file.txt" doesn't skip file if the target-file exists. This behaviour is consistent with old-distcp (which in turn adopts the behaviour of Unix cp.)

          2. DynamicInputChunk isn't public since it's specific to the implementation of DynamicInputFormat, and is as such not meant to be consumed except in conjunction with the DynamicInputFormat. I will change the FileSplit-size as you suggest, and remove the commented code.

          3. CopyCommitter:

          • You're right. Atomic-commit removing existing directories is not intuitive. I've changed that. Testing.
          • preserveFileAttributes() should have been named preserveFileAttributesOnDirectory(). I'll change that.
          • Will fix up the javadoc.

          I'll post a revised patch shortly.

          Show
          Mithun Radhakrishnan added a comment - A couple of notes on the comments: 0. In its current state, DistCpV2 is maven-ized. I'm considering not switching to ant and then switching back. 1. CopyMapper behaviour: The quoted check is for overwriting a file when the source-path and the target-path are both complete paths to pre-existing files. "distcp /source/file.txt /target/file.txt" doesn't skip file if the target-file exists. This behaviour is consistent with old-distcp (which in turn adopts the behaviour of Unix cp.) 2. DynamicInputChunk isn't public since it's specific to the implementation of DynamicInputFormat, and is as such not meant to be consumed except in conjunction with the DynamicInputFormat. I will change the FileSplit-size as you suggest, and remove the commented code. 3. CopyCommitter: You're right. Atomic-commit removing existing directories is not intuitive. I've changed that. Testing. preserveFileAttributes() should have been named preserveFileAttributesOnDirectory(). I'll change that. Will fix up the javadoc. I'll post a revised patch shortly.
          Hide
          Amareshwari Sriramadasu added a comment -

          First of all, the code needs go into a contrib project. So, you need to regenerate the patch putting the code in contrib.
          Also, build environment needs changes. Will this be blocked on mavenization of MapReduce?

          Overall, design looks fine. Here are some comments on the code:

          • CopyMapper:
            •     if (targetFS.exists(targetFinalPath) && targetFS.isFile(targetFinalPath)) {
                    overWrite = true; // When target is an existing file, overwrite it.
                  }
              

              Target file is overwritten irrespective of overwrite configuration? why?

          • Dynamic*
            • DynamicInputChunk is not public?
            • DynamicInputFormat creates FileSplits with zero length. Instead should it be created with the size of chunk as the size of the split.
            • DynamicRecordReader has commented code. Should remove it.
          • CopyCommitter:
            • Atomic commit should not delete the final directory. Should throw out an error if it exists even before starting the job.
            • deleteMissing() counts the files which do not exists at both source and target paths as deleted entries.
            • Preserving status for the root folder does not happen at all? Can you check?
            • If I’m not wrong, preserveFileAttributes() does preserve only for directories. Can we rename the method accordingly?
            • The methods deleteMissing(), preserveFileAttributes() etc need more doc.
            • Deleting attempt temp files happens in each attempt. Why are we doing delete again in Committer? Committer should just delete the work path.

          General comment:
          All public classes and public methods need javadoc

          Haven't looked at testcases.

          Show
          Amareshwari Sriramadasu added a comment - First of all, the code needs go into a contrib project. So, you need to regenerate the patch putting the code in contrib. Also, build environment needs changes. Will this be blocked on mavenization of MapReduce? Overall, design looks fine. Here are some comments on the code: CopyMapper: if (targetFS.exists(targetFinalPath) && targetFS.isFile(targetFinalPath)) { overWrite = true; // When target is an existing file, overwrite it. } Target file is overwritten irrespective of overwrite configuration? why? Dynamic* DynamicInputChunk is not public? DynamicInputFormat creates FileSplits with zero length. Instead should it be created with the size of chunk as the size of the split. DynamicRecordReader has commented code. Should remove it. CopyCommitter: Atomic commit should not delete the final directory. Should throw out an error if it exists even before starting the job. deleteMissing() counts the files which do not exists at both source and target paths as deleted entries. Preserving status for the root folder does not happen at all? Can you check? If I’m not wrong, preserveFileAttributes() does preserve only for directories. Can we rename the method accordingly? The methods deleteMissing(), preserveFileAttributes() etc need more doc. Deleting attempt temp files happens in each attempt. Why are we doing delete again in Committer? Committer should just delete the work path. General comment: All public classes and public methods need javadoc Haven't looked at testcases.
          Amareshwari Sriramadasu made changes -
          Assignee Mithun Radhakrishnan [ mithun ]
          Hide
          Mithun Radhakrishnan added a comment -

          Code's up on Github as well, if that works better for some.
          (https://github.com/mithunr/DistCpV2-0.20.203)

          Show
          Mithun Radhakrishnan added a comment - Code's up on Github as well, if that works better for some. ( https://github.com/mithunr/DistCpV2-0.20.203 )
          Mithun Radhakrishnan made changes -
          Description This is a slightly modified version of the DistCp rewrite that Yahoo(!) uses in production today. The rewrite was ground-up, with specific focus on:
          1. improved startup time (postponing as much work as possible to the MR job)
          2. support for multiple copy-strategies
          3. new features (e.g. -atomic, -async, -bandwidth.)
          4. improved programmatic use
          Some effort has gone into refactoring what used to be achieved by a single large (1.7 KLOC) source file, into a design that (hopefully) reads better too.

          The proposed DistCpV2 preserves command-line-compatibility with the old version, and should be a drop-in replacement.

          New to v2:

          1. Copy-strategies and the DynamicInputFormat:
          A copy-strategy determines the policy by which source-file-paths are distributed between map-tasks. (These boil down to the choice of the input-format.)
          If no strategy is explicitly specified on the command-line, the policy chosen is "uniform size", where v2 behaves identically to old-DistCp. (The number of bytes transferred by each map-task is roughly equal, at a per-file granularity.)
          Alternatively, v2 ships with a "dynamic" copy-strategy (in the DynamicInputFormat). This policy acknowledges that
          (i) dividing files based only on file-size might not be an even distribution (E.g. if some datanodes are slower than others, or if some files are skipped.)
          (ii) a "static" association of a source-path to a map increases the likelihood of long-tails during copy.
          The "dynamic" strategy divides the list-of-source-paths into a number (> nMaps) of smaller parts. When each map completes its current list of paths, it picks up a new list to process, if available. So if a map-task is stuck on a slow (and not necessarily large) file, other maps can pick up the slack. The thinner the file-list is sliced, the greater the parallelism (and the lower the chances of long-tails). Within reason, of course: the number of these short-lived list-files is capped at an overridable maximum.
          Internal benchmarks against source/target clusters with some slow(ish) datanodes have indicated significant performance gains when using the dynamic-strategy. Gains are most pronounced when nFiles greatly exceeds nMaps.
          Please note that the DynamicInputFormat might prove useful outside of DistCp. It is hence available as a mapred/lib, unfettered to DistCpV2. Also note that the copy-strategies have no bearing on the CopyMapper.map() implementation.

          2. Improved startup-time and programmatic use:
          When the old-DistCp runs with -update, and creates the list-of-source-paths, it attempts to filter out files that might be skipped (by comparing file-sizes, checksums, etc.) This significantly increases the startup time (or the time spent in serial processing till the MR job is launched), blocking the calling-thread. This becomes pronounced as nFiles increases. (Internal benchmarks have seen situations where more time is spent setting up the job than on the actual transfer.)
          DistCpV2 postpones as much work as possible to the MR job. The file-listing isn't filtered until the map-task runs (at which time, identical files are skipped). DistCpV2 can now be run "asynchronously". The program quits at job-launch, logging the job-id for tracking. Programmatically, the DistCp.execute() returns a Job instance for progress-tracking.

          3. New features:
          (i) -async: As described in #2.
          (ii) -atomic: Data is copied to a (user-specifiable) tmp-location, and then moved atomically to destination.
          (iii) -bandwidth: Enforces a limit on the bandwidth consumed per map.
          (iv) -strategy: As above.

          A more comprehensive description the newer features, how the dynamic-strategy works, etc. is available in src/site/xdoc/, and in the pdf that's generated therefrom, during the build.

          I look forward to comments, suggestions and discussion that will hopefully ensue. I have this running against Hadoop 0.20.203.0. I also have a port to 0.23.0 (complete with unit-tests).

          P.S.
          A tip of the hat to Srikanth (Sundarrajan) and Venkatesh (Seetharamaiah), for ideas, code, reviews and guidance. Although much of the code is mine, the idea to use the DFS to implement "dynamic" input-splits wasn't.

          This is a slightly modified version of the DistCp rewrite that Yahoo uses in production today. The rewrite was ground-up, with specific focus on:
          1. improved startup time (postponing as much work as possible to the MR job)
          2. support for multiple copy-strategies
          3. new features (e.g. -atomic, -async, -bandwidth.)
          4. improved programmatic use
          Some effort has gone into refactoring what used to be achieved by a single large (1.7 KLOC) source file, into a design that (hopefully) reads better too.

          The proposed DistCpV2 preserves command-line-compatibility with the old version, and should be a drop-in replacement.

          New to v2:

          1. Copy-strategies and the DynamicInputFormat:
          A copy-strategy determines the policy by which source-file-paths are distributed between map-tasks. (These boil down to the choice of the input-format.)
          If no strategy is explicitly specified on the command-line, the policy chosen is "uniform size", where v2 behaves identically to old-DistCp. (The number of bytes transferred by each map-task is roughly equal, at a per-file granularity.)
          Alternatively, v2 ships with a "dynamic" copy-strategy (in the DynamicInputFormat). This policy acknowledges that
          (a) dividing files based only on file-size might not be an even distribution (E.g. if some datanodes are slower than others, or if some files are skipped.)
          (b) a "static" association of a source-path to a map increases the likelihood of long-tails during copy.
          The "dynamic" strategy divides the list-of-source-paths into a number (> nMaps) of smaller parts. When each map completes its current list of paths, it picks up a new list to process, if available. So if a map-task is stuck on a slow (and not necessarily large) file, other maps can pick up the slack. The thinner the file-list is sliced, the greater the parallelism (and the lower the chances of long-tails). Within reason, of course: the number of these short-lived list-files is capped at an overridable maximum.
          Internal benchmarks against source/target clusters with some slow(ish) datanodes have indicated significant performance gains when using the dynamic-strategy. Gains are most pronounced when nFiles greatly exceeds nMaps.
          Please note that the DynamicInputFormat might prove useful outside of DistCp. It is hence available as a mapred/lib, unfettered to DistCpV2. Also note that the copy-strategies have no bearing on the CopyMapper.map() implementation.

          2. Improved startup-time and programmatic use:
          When the old-DistCp runs with -update, and creates the list-of-source-paths, it attempts to filter out files that might be skipped (by comparing file-sizes, checksums, etc.) This significantly increases the startup time (or the time spent in serial processing till the MR job is launched), blocking the calling-thread. This becomes pronounced as nFiles increases. (Internal benchmarks have seen situations where more time is spent setting up the job than on the actual transfer.)
          DistCpV2 postpones as much work as possible to the MR job. The file-listing isn't filtered until the map-task runs (at which time, identical files are skipped). DistCpV2 can now be run "asynchronously". The program quits at job-launch, logging the job-id for tracking. Programmatically, the DistCp.execute() returns a Job instance for progress-tracking.

          3. New features:
          (a) -async: As described in #2.
          (b) -atomic: Data is copied to a (user-specifiable) tmp-location, and then moved atomically to destination.
          (c) -bandwidth: Enforces a limit on the bandwidth consumed per map.
          (d) -strategy: As above.

          A more comprehensive description the newer features, how the dynamic-strategy works, etc. is available in src/site/xdoc/, and in the pdf that's generated therefrom, during the build.

          High on the list of things to do is support to parallelize copies on a per-block level. (i.e. Incorporation of HDFS-222.)

          I look forward to comments, suggestions and discussion that will hopefully ensue. I have this running against Hadoop 0.20.203.0. I also have a port to 0.23.0 (complete with unit-tests).

          P.S.
          A tip of the hat to Srikanth (Sundarrajan) and Venkatesh (Seetharamaiah), for ideas, code, reviews and guidance. Although much of the code is mine, the idea to use the DFS to implement "dynamic" input-splits wasn't.

          Mithun Radhakrishnan made changes -
          Summary DistCpV2: Rewrite to improve startup time, support multiple copy-strategies (including the "DynamicInputFormat"), improve copy performance and to improve code readability. DistCp Rewrite
          Mithun Radhakrishnan made changes -
          Field Original Value New Value
          Attachment distcpv2.20.203.patch [ 12488877 ]
          Mithun Radhakrishnan created issue -

            People

            • Assignee:
              Mithun Radhakrishnan
              Reporter:
              Mithun Radhakrishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development