Hadoop Common
  1. Hadoop Common
  2. HADOOP-6558

archive does not work with distcp -update

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following distcp command works.

      hadoop distcp -Dmapred.job.queue.name=q har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101 t101_distcp
      

      However, it does not work for -update.

      -bash-3.1$ hadoop distcp -Dmapred.job.queue.name=q -update har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101 t101_distcp
      10/01/29 20:06:53 INFO tools.DistCp: srcPaths=[har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101]
      10/01/29 20:06:53 INFO tools.DistCp: destPath=t101
      java.lang.IllegalArgumentException: Wrong FS: har://hdfs-nn_hostname:8020/user/tsz/t101.har/t101/text-00000000, expected: hdfs://nn_hostname
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
              at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:463)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:46)
              at org.apache.hadoop.fs.FilterFileSystem.getFileChecksum(FilterFileSystem.java:250)
              at org.apache.hadoop.tools.DistCp.sameFile(DistCp.java:1204)
              at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1084)
              ...
      
      1. c6558_20100216.patch
        0.6 kB
        Tsz Wo Nicholas Sze
      2. c6558_20100216b_y0.20.patch
        1 kB
        Tsz Wo Nicholas Sze
      3. c6558_20100216b.patch
        1 kB
        Tsz Wo Nicholas Sze

        Activity

        Tsz Wo Nicholas Sze created issue -
        Mahadev konar made changes -
        Field Original Value New Value
        Fix Version/s 0.22.0 [ 12314184 ]
        Tsz Wo Nicholas Sze made changes -
        Project Hadoop Map/Reduce [ 12310941 ] Hadoop Common [ 12310240 ]
        Key MAPREDUCE-1431 HADOOP-6558
        Fix Version/s 0.22.0 [ 12314296 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Component/s fs [ 12310689 ]
        Component/s harchive [ 12312903 ]
        Tsz Wo Nicholas Sze made changes -
        Attachment c6558_20100216.patch [ 12436032 ]
        Tsz Wo Nicholas Sze made changes -
        Attachment c6558_20100216b.patch [ 12436084 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Tsz Wo Nicholas Sze made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Assignee Mahadev konar [ mahadev ] Tsz Wo (Nicholas), SZE [ szetszwo ]
        Resolution Fixed [ 1 ]
        Tsz Wo Nicholas Sze made changes -
        Attachment c6558_20100216b_y0.20.patch [ 12436264 ]
        Tom White made changes -
        Fix Version/s 0.21.0 [ 12313563 ]
        Fix Version/s 0.22.0 [ 12314296 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development