[HADOOP-4359] Access Token: Support for data access authorization checking on DataNodes

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.0
• Fix Version/s: 0.21.0
• Hadoop Flags: Reviewed
• Release Note: Introduced access tokens as capabilities for accessing datanodes. This change to internal protocols does not affect client applications.
• Assignee: Kan Zhang
• Reporter: Kan Zhang
• Created: 07/Oct/08 04:02
• Updated: 24/Aug/10 20:34
• Resolved: 13/May/09 21:30

Description

Currently, DataNodes do not enforce any access control on accesses to its data blocks. This makes it possible for an unauthorized client to read a data block as long as she can supply its block ID. It's also possible for anyone to write arbitrary data blocks to DataNodes.

When users request file accesses on the NameNode, file permission checking takes place. Authorization decisions are made with regard to whether the requested accesses to those files (and implicitly, to their corresponding data blocks) are permitted. However, when it comes to subsequent data block accesses on the DataNodes, those authorization decisions are not made available to the DataNodes and consequently, such accesses are not verified. Datanodes are not capable of reaching those decisions independently since they don't have concepts of files, let alone file permissions.

In order to implement data access policies consistently across HDFS services, there is a need for a mechanism by which authorization decisions made on the NameNode can be faithfully enforced on the DataNodes and any unauthorized access is declined.

Issue Links

• is related to: HADOOP-5824 remove OP_READ_METADATA functionality from Datanode
• is related to: HDFS-195 Need to handle access token expiration when re-establishing the pipeline for dfs write
• is related to: HADOOP-4487 Security features for Hadoop
• is related to: HDFS-265 Revisit append
• is related to: HDFS-409 Add more access token tests
• relates to: HDFS-992 Re-factor block access token implementation to conform to the generic Token interface in Common