Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.6.5
    • Fix Version/s: None
    • Component/s: tools/distcp
    • Labels:
      None
    • Target Version/s:

      Description

      When using distcp to copy lots of small files, NameNode naturally becomes a bottleneck.

      The current distcp code did not optimize to reduce the NameNode calls. We should restructure the code to reduce the number of NameNode calls as much as possible to speed up the copy of small files.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zshao Zheng Shao
                Reporter:
                zshao Zheng Shao
              • Votes:
                1 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated: