Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6414

Distcp command very slow to enumerate files needing

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.5.0
    • Fix Version/s: None
    • Component/s: distcp
    • Labels:
      None
    • Environment:

      RHEL 6.5

      Description

      When copying large amounts of data using distcp utility (100's of TBs), the distcp utility takes a large time to enumerate all of the files that have changed. In my system, this corresponds to 14-16 hours before the actual copying of data begins.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              thale013 Tyler Hale
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: