Issue Details (XML | Word | Printable)

Key: MAPREDUCE-842
Type: Sub-task Sub-task
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Vinod K V
Reporter: Arun C Murthy
Votes: 1
Watchers: 8
Operations

If you were logged in you would be able to see more operations.
Hadoop Map/Reduce
MAPREDUCE-563

Per-job local data on the TaskTracker node should have right access-control

Created: 22/Oct/08 08:27 PM   Updated: 22/Sep/09 06:44 AM
Return to search
Component/s: tasktracker
Affects Version/s: None
Fix Version/s: 0.21.0

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works HADOOP-4491-20090623-common.1.txt 2009-06-23 08:05 AM Vinod K V 24 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090623-mapred.1.txt 2009-06-23 08:05 AM Vinod K V 104 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090703-common.1.txt 2009-07-03 11:41 AM Vinod K V 23 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090703-common.txt 2009-07-03 10:39 AM Vinod K V 23 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090703.1.txt 2009-07-03 11:41 AM Vinod K V 136 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090703.txt 2009-07-03 10:39 AM Vinod K V 139 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090707-common.txt 2009-07-07 04:59 AM Vinod K V 11 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090707.txt 2009-07-07 04:59 AM Vinod K V 137 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090716-mapred.txt 2009-07-16 07:02 AM Vinod K V 134 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090803.1.txt 2009-08-03 10:13 AM Vinod K V 170 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090803.txt 2009-08-03 05:13 AM Vinod K V 171 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090806.txt 2009-08-06 11:37 AM Vinod K V 190 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090807.2.txt 2009-08-07 06:45 AM Vinod K V 194 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090810.1.txt 2009-08-10 07:02 AM Vinod K V 201 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090810.3.txt 2009-08-10 10:21 AM Vinod K V 201 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090811.txt 2009-08-11 04:16 AM Vinod K V 205 kB
Text File Licensed for inclusion in ASF works HADOOP-4491-20090812.txt 2009-08-12 08:27 AM Vinod K V 205 kB
Issue Links:
Duplicate
 
Reference
 

Hadoop Flags: Reviewed
Release Note:
Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes:
 - All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group.
 - The permissions of the file/directories are set to the most restrictive permissions possible.
 - Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization.
 - Also fixes MAPREDUCE-131 which is related to a bug because of which tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir
 - A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg so as to support restricted access control for userlogs of the tasks on the TaskTracker.
Resolution Date: 12/Aug/09 04:21 PM


 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #803583 Wed Aug 12 16:17:47 UTC 2009 yhemanth MAPREDUCE-842. Setup secure permissions for localized job files, intermediate outputs and log files on tasktrackers. Contributed by Vinod Kumar Vavilapalli.
Files Changed
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/ClusterWithLinuxTaskController.java
MODIFY /hadoop/mapreduce/trunk/CHANGES.txt
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/LinuxTaskController.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/Makefile.in
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/MapTaskRunner.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/configuration.h.in
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/ReduceTaskRunner.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestMapRed.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/configure.ac
ADD /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestTaskTrackerLocalization.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/main.c
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/task-controller.c
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/TaskRunner.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/DefaultTaskController.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JobConf.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/MapTask.java
MODIFY /hadoop/mapreduce/trunk/src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestUlimit.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/TaskLog.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/BackupStore.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/task-controller.h
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/IsolationRunner.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/Child.java
ADD /hadoop/mapreduce/trunk/src/c++/task-controller/tests
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestIsolationRunner.java
MODIFY /hadoop/mapreduce/trunk/src/contrib/build-contrib.xml
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/Task.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/TaskTracker.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestJobExecutionAsDifferentUser.java
MODIFY /hadoop/mapreduce/trunk/src/c++/task-controller/configuration.c
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/LocalJobRunner.java
ADD /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestLocalizationWithLinuxTaskController.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestKillSubProcesses.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/TaskController.java
MODIFY /hadoop/mapreduce/trunk/src/docs/src/documentation/content/xdocs/cluster_setup.xml
MODIFY /hadoop/mapreduce/trunk/src/contrib/streaming/src/test/org/apache/hadoop/streaming/TestStreamingAsDifferentUser.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/MiniMRCluster.java
MODIFY /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestMiniMRWithDFS.java
ADD /hadoop/mapreduce/trunk/src/c++/task-controller/tests/test-task-controller.c
MODIFY /hadoop/mapreduce/trunk/build.xml
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JvmManager.java
MODIFY /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/MapOutputFile.java
MODIFY /hadoop/mapreduce/trunk/conf/taskcontroller.cfg