Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2228

Jobs fail because job.xml exists

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.3
    • Fix Version/s: 0.15.2, 0.16.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      35 node cluster, linux

      Description

      org.apache.hadoop.ipc.RemoteException: java.io.IOException: Target /var/storage/4/mapred/local/jobTracker/job_200711081903_3976.xml already exists
      at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:271)
      at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:117)
      at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:803)
      at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:784)
      at org.apache.hadoop.mapred.JobInProgress.<init>(JobInProgress.java:134)
      at org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:1479)
      at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566)

      at org.apache.hadoop.ipc.Client.call(Client.java:470)
      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165)
      at $Proxy1.submitJob(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
      at $Proxy1.submitJob(Unknown Source)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:397)
      at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:345)
      at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:250)
      at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:282)
      at java.lang.Thread.run(Thread.java:619)

      Perhaps related to HADOOP-1057, HADOOP-891 or to the rpc retry. It seems my job was submitted and actually finished despite the exception. Could it be that the job went in and the rpc retry decided to submit it again anyway?

        Attachments

        1. HADOOP-2228-v1.patch
          1.0 kB
          Johan Oskarsson

          Activity

            People

            • Assignee:
              johanoskarsson Johan Oskarsson
              Reporter:
              johanoskarsson Johan Oskarsson
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: