Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3079

usercache/<user>/appcache/<appid> directory not removed when using DefaultContainerExecutor

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.24.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      Running with the DefaultContainerExecutor it appears that the usercache/<user>/appcache/<appid> directory itself is not removed when the app finishes. All the directories under it are properly removed though.

      The nodemanager log file indicates that it tries to delete it:
      11/09/23 15:17:56 INFO nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoop/mapred/tmp/mapred-local/usercache/tgraves/appcache/application_1316722920862_0003

      This doesn't appear to happen with the LinuxContainerExecutor.

        Activity

        Hide
        Siddharth Seth added a comment -

        https://issues.apache.org/jira/browse/MAPREDUCE-2751 added a log message when the the DCE fails to delete a file (using the FileContext API). This could happen with the container dir or the app dir. We could either retry or try figuring out why the filecontext api is failing to delete the file.

        Show
        Siddharth Seth added a comment - https://issues.apache.org/jira/browse/MAPREDUCE-2751 added a log message when the the DCE fails to delete a file (using the FileContext API). This could happen with the container dir or the app dir. We could either retry or try figuring out why the filecontext api is failing to delete the file.
        Hide
        Todd Lipcon added a comment -

        I'm not actively working on this, so if someone else needs it done, feel free to take it.

        Show
        Todd Lipcon added a comment - I'm not actively working on this, so if someone else needs it done, feel free to take it.
        Hide
        Jason Lowe added a comment -

        Is this still an issue? I was unable to reproduce it on 0.23.3 or trunk.

        Show
        Jason Lowe added a comment - Is this still an issue? I was unable to reproduce it on 0.23.3 or trunk.

          People

          • Assignee:
            Unassigned
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:

              Development