Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6558

archive does not work with distcp -update

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following distcp command works.

      hadoop distcp -Dmapred.job.queue.name=q har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101 t101_distcp
      

      However, it does not work for -update.

      -bash-3.1$ hadoop distcp -Dmapred.job.queue.name=q -update har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101 t101_distcp
      10/01/29 20:06:53 INFO tools.DistCp: srcPaths=[har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101]
      10/01/29 20:06:53 INFO tools.DistCp: destPath=t101
      java.lang.IllegalArgumentException: Wrong FS: har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101/text-00000000, expected: hdfs://nn_hostname
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
              at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:463)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:46)
              at org.apache.hadoop.fs.FilterFileSystem.getFileChecksum(FilterFileSystem.java:250)
              at org.apache.hadoop.tools.DistCp.sameFile(DistCp.java:1204)
              at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1084)
              ...
      

        Attachments

        1. c6558_20100216b.patch
          1 kB
          Tsz Wo Nicholas Sze
        2. c6558_20100216b_y0.20.patch
          1 kB
          Tsz Wo Nicholas Sze
        3. c6558_20100216.patch
          0.6 kB
          Tsz Wo Nicholas Sze

          Activity

            People

            • Assignee:
              szetszwo Tsz Wo Nicholas Sze
              Reporter:
              szetszwo Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: