Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3832

Resource Localization fails on a cluster due to existing cache directories

    Details

    • Hadoop Flags:
      Reviewed

      Description

      We have found resource localization fails on a cluster with following error.

      Got this error in hadoop-2.7.0 release which was fixed in 2.6.0 (YARN-2624)

      Application application_1434703279149_0057 failed 2 times due to AM Container for appattempt_1434703279149_0057_000002 exited with exitCode: -1000
      For more detailed output, check application tracking page:http://S0559LDPag68:45020/cluster/app/application_1434703279149_0057Then, click on links to logs of each attempt.
      Diagnostics: Rename cannot overwrite non empty destination directory /opt/hdfsdata/HA/nmlocal/usercache/root/filecache/39
      java.io.IOException: Rename cannot overwrite non empty destination directory /opt/hdfsdata/HA/nmlocal/usercache/root/filecache/39
      at org.apache.hadoop.fs.AbstractFileSystem.renameInternal(AbstractFileSystem.java:735)
      at org.apache.hadoop.fs.FilterFs.renameInternal(FilterFs.java:244)
      at org.apache.hadoop.fs.AbstractFileSystem.rename(AbstractFileSystem.java:678)
      at org.apache.hadoop.fs.FileContext.rename(FileContext.java:958)
      at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:366)
      at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:62)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Failing this attempt. Failing the application.
      

        Attachments

        1. YARN-3832.patch
          1 kB
          Brahma Reddy Battula

          Issue Links

            Activity

              People

              • Assignee:
                brahmareddy Brahma Reddy Battula
                Reporter:
                singar.ranga Ranga Swamy
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: