Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 0.20.205.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      $ hadoop distcp hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000 /user/hadoopqa/out3
      11/09/30 18:57:59 INFO tools.DistCp: srcPaths=[hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000]
      11/09/30 18:57:59 INFO tools.DistCp: destPath=/user/hadoopqa/out3
      11/09/30 18:58:00 INFO security.TokenCache: Got dt for
      hftp://<NN Hostname>:50070/user/hadoopqa/input1/part-00000;uri=<NN IP>:50470;t.service=<NN IP>:50470
      11/09/30 18:58:00 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 24 for hadoopqa on <NN IP>:8020
      11/09/30 18:58:00 INFO security.TokenCache: Got dt for
      /user/hadoopqa/out3;uri=<NN IP>:8020;t.service=<NN IP>:8020
      11/09/30 18:58:00 INFO tools.DistCp: /user/hadoopqa/out3 does not exist.
      11/09/30 18:58:00 INFO tools.DistCp: sourcePathsCount=1
      11/09/30 18:58:00 INFO tools.DistCp: filesToCopyCount=1
      11/09/30 18:58:00 INFO tools.DistCp: bytesToCopyCount=1.0g
      11/09/30 18:58:01 INFO mapred.JobClient: Running job: job_201109300819_0007
      11/09/30 18:58:02 INFO mapred.JobClient: map 0% reduce 0%
      11/09/30 18:58:25 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_0, Status : FAILED
      java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
      at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)

      11/09/30 18:58:41 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_1, Status : FAILED
      java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
      at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)

      11/09/30 18:58:56 INFO mapred.JobClient: Task Id : attempt_201109300819_0007_m_000000_2, Status : FAILED
      java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
      at org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
      at org.apache.hadoop.mapred.Child.main(Child.java:249)

      11/09/30 18:59:14 INFO mapred.JobClient: Job complete: job_201109300819_0007
      11/09/30 18:59:14 INFO mapred.JobClient: Counters: 6
      11/09/30 18:59:14 INFO mapred.JobClient: Job Counters
      11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=62380
      11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
      11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
      11/09/30 18:59:14 INFO mapred.JobClient: Launched map tasks=4
      11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
      11/09/30 18:59:14 INFO mapred.JobClient: Failed map tasks=1
      11/09/30 18:59:14 INFO mapred.JobClient: Job Failed: # of failed Map Tasks exceeded allowed limit. FailedCount: 1.
      LastFailedTask: task_201109300819_0007_m_000000
      With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.io.IOException: Job failed!
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1257)
      at org.apache.hadoop.tools.DistCp.copy(DistCp.java:667)
      at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                rajsaha Rajit Saha
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: