[HADOOP-9709] Add ability in Hadoop servers (Namenode, Datanode, ResourceManager ) to support multiple QOP (Authentication , Privacy)

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Benoy Antony
• Reporter: Benoy Antony
• Due: 26/Jul/13
• Created: 08/Jul/13 21:52
• Updated: 31/Mar/14 23:39
• Resolved: 31/Mar/14 23:35

Description

Hadoop Servers currently support only one QOP for the whole cluster.
We want Hadoop servers to support different quality of protection at the same time. This will enable different clients to use different QOP.

A simple usecase:

Let each Hadoop server support two QOP .
1. Authentication
2. Privacy (Privacy includes Authentication) .

The Hadoop servers and internal clients does Authentication without incurring cost of encryption. External clients use Privacy.
The hadoop servers and internal clients are inside the firewall. External clients are outside the firewall.

As an enhancement , it is possible to add a pluggable check (eg. IP whitelist) to identify internal and external clients.

The implementation is simple.
Each Hadoop server listens on multiple ports by configuration with different QOP.

For the usecase mentioned above, the servers - NameNode, DataNode, ResourceManager listen on two ports (much like 80(http) and 443(https)) for RPC and Streaming. ApplicationMaster uses a range of ports for privacy and non-privacy and picks up a port and QOP based on client's config for client communication.

The clients specify the port which they are supposed to connect to. Clients specify the rpc protection as well encryption policy for streaming layer.

This is an umbrella jira .
I have divided this feature into multiple small tasks. I'll add testcases once the approach is reviewed.

Issue Links

• relates to: HADOOP-10057 Add ability in Hadoop servers (Namenode, JobTracker, Datanode ) to support multiple QOP (Authentication , Privacy) simultaneously
• requires: HADOOP-9710 Modify security layer to support QoP based on ports
• requires: HDFS-4964 Enable multiple QOP for NameNode
• requires: HDFS-4965 Make datanodes support multiple QOP for RPC and streaming protocols
• requires: MAPREDUCE-5378 Enable ApplicationMaster to support different QOP for client and server communications
• requires: YARN-904 Enable multiple QOP for ResourceManager