[#MAPREDUCE-842] Per-job local data on the TaskTracker node should have right access-control

Hadoop Map/Reduce

MAPREDUCE-563

Per-job local data on the TaskTracker node should have right access-control

Created: 22/Oct/08 08:27 PM Updated: 22/Sep/09 06:44 AM

Return to search

Component/s:

tasktracker

Affects Version/s:

None

Fix Version/s:

0.21.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
HADOOP-4491-20090623-common.1.txt	2009-06-23 08:05 AM	Vinod K V	24 kB
HADOOP-4491-20090623-mapred.1.txt	2009-06-23 08:05 AM	Vinod K V	104 kB
HADOOP-4491-20090703-common.1.txt	2009-07-03 11:41 AM	Vinod K V	23 kB
HADOOP-4491-20090703-common.txt	2009-07-03 10:39 AM	Vinod K V	23 kB
HADOOP-4491-20090703.1.txt	2009-07-03 11:41 AM	Vinod K V	136 kB
HADOOP-4491-20090703.txt	2009-07-03 10:39 AM	Vinod K V	139 kB
HADOOP-4491-20090707-common.txt	2009-07-07 04:59 AM	Vinod K V	11 kB
HADOOP-4491-20090707.txt	2009-07-07 04:59 AM	Vinod K V	137 kB
HADOOP-4491-20090716-mapred.txt	2009-07-16 07:02 AM	Vinod K V	134 kB
HADOOP-4491-20090803.1.txt	2009-08-03 10:13 AM	Vinod K V	170 kB
HADOOP-4491-20090803.txt	2009-08-03 05:13 AM	Vinod K V	171 kB
HADOOP-4491-20090806.txt	2009-08-06 11:37 AM	Vinod K V	190 kB
HADOOP-4491-20090807.2.txt	2009-08-07 06:45 AM	Vinod K V	194 kB
HADOOP-4491-20090810.1.txt	2009-08-10 07:02 AM	Vinod K V	201 kB
HADOOP-4491-20090810.3.txt	2009-08-10 10:21 AM	Vinod K V	201 kB
HADOOP-4491-20090811.txt	2009-08-11 04:16 AM	Vinod K V	205 kB
HADOOP-4491-20090812.txt	2009-08-12 08:27 AM	Vinod K V	205 kB

Issue Links:

Duplicate

This issue duplicates:

~~MAPREDUCE-131~~ After HADOOP-5420, tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir

Reference

This issue relates to:

~~MAPREDUCE-720~~ Task logs should have right access-control

Hadoop Flags:	Reviewed
Release Note:	« Hide Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes: - All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group. - The permissions of the file/directories are set to the most restrictive permissions possible. - Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization. - Also fixes ~~MAPREDUCE-131~~ which is related to a bug because of which tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir - A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg so as to support restricted access control for userlogs of the tasks on the TaskTracker. Show » Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes: - All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group. - The permissions of the file/directories are set to the most restrictive permissions possible. - Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization. - Also fixes ~~MAPREDUCE-131~~ which is related to a bug because of which tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir - A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg so as to support restricted access control for userlogs of the tasks on the TaskTracker.
Resolution Date:	12/Aug/09 04:21 PM

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Arun C Murthy made changes - 22/Oct/08 08:37 PM

Field	Original Value	New Value
Component/s		security [ 12312526 ]
Component/s		mapred [ 12310690 ]

Hemanth Yamijala made changes - 31/Oct/08 06:44 AM

Assignee

Arun C Murthy [ acmurthy ]

Vinod K V made changes - 05/Jun/09 11:18 AM

Assignee

Vinod K V [ vinodkv ]

Vinod K V made changes - 23/Jun/09 08:05 AM

Attachment		HADOOP-4491-20090623-mapred.1.txt [ 12411506 ]
Attachment		HADOOP-4491-20090623-common.1.txt [ 12411505 ]

Vinod K V made changes - 23/Jun/09 08:26 AM

Link

This issue duplicates ~~MAPREDUCE-131~~ [ ~~MAPREDUCE-131~~ ]

Vinod K V made changes - 26/Jun/09 07:50 AM

Link

This issue is blocked by ~~HADOOP-6111~~ [ ~~HADOOP-6111~~ ]

Vinod K V made changes - 03/Jul/09 10:39 AM

Attachment		HADOOP-4491-20090703-common.txt [ 12412474 ]
Attachment		HADOOP-4491-20090703.txt [ 12412473 ]

Vinod K V made changes - 03/Jul/09 11:41 AM

Attachment		HADOOP-4491-20090703-common.1.txt [ 12412483 ]
Attachment		HADOOP-4491-20090703.1.txt [ 12412482 ]

Vinod K V made changes - 07/Jul/09 04:59 AM

Attachment		HADOOP-4491-20090707.txt [ 12412704 ]
Attachment		HADOOP-4491-20090707-common.txt [ 12412703 ]

Vinod K V made changes - 07/Jul/09 05:13 AM

Link

This issue relates to ~~MAPREDUCE-720~~ [ ~~MAPREDUCE-720~~ ]

Owen O'Malley made changes - 08/Jul/09 04:53 PM

Component/s

mapred [ 12310690 ]

Vinod K V made changes - 16/Jul/09 07:02 AM

Attachment

HADOOP-4491-20090716-mapred.txt [ 12413652 ]

Vinod K V made changes - 03/Aug/09 05:13 AM

Attachment

HADOOP-4491-20090803.txt [ 12415309 ]

Vinod K V made changes - 03/Aug/09 10:13 AM

Attachment

HADOOP-4491-20090803.1.txt [ 12415343 ]

Vinod K V made changes - 06/Aug/09 11:37 AM

Attachment

HADOOP-4491-20090806.txt [ 12415730 ]

Vinod K V made changes - 07/Aug/09 06:45 AM

Attachment

HADOOP-4491-20090807.2.txt [ 12415824 ]

Hemanth Yamijala made changes - 10/Aug/09 05:22 AM

Link

This issue is blocked by ~~HADOOP-6111~~ [ ~~HADOOP-6111~~ ]

Vinod K V made changes - 10/Aug/09 07:02 AM

Attachment

HADOOP-4491-20090810.1.txt [ 12416027 ]

Vinod K V made changes - 10/Aug/09 07:03 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

Vinod K V made changes - 10/Aug/09 10:18 AM

Status

Patch Available [ 10002 ]

Open [ 1 ]

Vinod K V made changes - 10/Aug/09 10:21 AM

Attachment

HADOOP-4491-20090810.3.txt [ 12416038 ]

Vinod K V made changes - 10/Aug/09 11:54 AM

Issue Type	Sub-task [ 7 ]	Bug [ 1 ]
Parent	HADOOP-4487 [ 12407018 ]

Vinod K V made changes - 10/Aug/09 11:55 AM

Component/s	security [ 12312526 ]
Component/s		tasktracker [ 12312906 ]
Key	HADOOP-4491	~~MAPREDUCE-842~~
Project	Hadoop Common [ 12310240 ]	Hadoop Map/Reduce [ 12310941 ]

Vinod K V made changes - 10/Aug/09 11:56 AM

Issue Type	Bug [ 1 ]	Sub-task [ 7 ]
Parent		MAPREDUCE-563 [ 12428534 ]

Vinod K V made changes - 10/Aug/09 11:57 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

Vinod K V made changes - 11/Aug/09 04:15 AM

Status

Patch Available [ 10002 ]

Open [ 1 ]

Vinod K V made changes - 11/Aug/09 04:16 AM

Attachment

HADOOP-4491-20090811.txt [ 12416158 ]

Vinod K V made changes - 11/Aug/09 04:17 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

Vinod K V made changes - 12/Aug/09 08:27 AM

Attachment

HADOOP-4491-20090812.txt [ 12416292 ]

Vinod K V made changes - 12/Aug/09 08:28 AM

Status

Patch Available [ 10002 ]

Open [ 1 ]

Vinod K V made changes - 12/Aug/09 08:29 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

Hemanth Yamijala made changes - 12/Aug/09 04:21 PM

Resolution		Fixed [ 1 ]
Fix Version/s		0.21.0 [ 12314045 ]
Hadoop Flags		[Reviewed]
Status	Patch Available [ 10002 ]	Resolved [ 5 ]

Vinod K V made changes - 22/Sep/09 06:40 AM

Release Note

Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes:
- All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group.
- The permissions of the file/directories are set to the most restrictive permissions possible.
- Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization.
- A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg

Vinod K V made changes - 22/Sep/09 06:44 AM

Release Note

Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes:
- All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group.
- The permissions of the file/directories are set to the most restrictive permissions possible.
- Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization.
- Also fixes ~~MAPREDUCE-131~~ which is related to a bug because of which tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir
- A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg so as to support restricted access control for userlogs of the tasks on the TaskTracker.