Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4246

Failure in deleting user directories in Secure hadoop

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.22.1
    • security
    • None
    • Reviewed

    Description

      This happens when security is enabled on 22

      When TaskTracker starts p, it invokes MRAsyncDiskService moves the contents of scratch/tasktracker to toBeDeleted under a directory created with the current timestamp.
      The owner of this directory is hadoop (owner of tasktracker)

      The contents of this directory are various usernames and they are owned by individual users. But MRAsyncDiskService tries to delete the newly created directory as hadoop and it fails.

      2012-05-01 16:04:58,805 DEBUG org.apache.hadoop.mapred.LinuxTaskController: deleteAsUser: [/apache/hadoop-assemble-argon-0.228/hadoop-0.22-argon-0.105/bin/../bin/task-controller, hadoop, 3, /hadoop00/scratch/toBeDeleted/2012-04-17_16-22-03.965_0]
      2012-05-01 16:04:58,809 WARN org.apache.hadoop.mapreduce.util.MRAsyncDiskService: Failure in deletion of toBeDeleted/2012-04-17_16-22-03.965_0 on /hadoop00/scratch with original name /hadoop00/scratch/toBeDeleted/2012-04-17_16-22-03.965_0 with exception org.apache.hadoop.util.Shell$ExitCodeException:
      at org.apache.hadoop.util.Shell.runCommand(Shell.java:256)
      at org.apache.hadoop.util.Shell.run(Shell.java:183)
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:376)
      at org.apache.hadoop.mapred.LinuxTaskController.deleteAsUser(LinuxTaskController.java:273)
      at org.apache.hadoop.mapreduce.util.MRAsyncDiskService$DeleteTask.run(MRAsyncDiskService.java:237)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)

      Attachments

        1. cleanup-user-dir.patch
          11 kB
          Benoy Antony

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            benoyantony Benoy Antony
            benoyantony Benoy Antony
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment