[GIRAPH-211] Add secure authentication to Netty IPC

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Assignee: Eugene Joseph Koontz
• Reporter: Eugene Joseph Koontz
• Created: 20/Jun/12 22:19
• Updated: 13/Feb/14 18:15
• Resolved: 10/Oct/12 20:17

Description

Gianmarco De Francisci Morales asked on the user list:

I am getting the exception in the subject when running my giraph program

on a cluster with Kerberos authentication.

This leads to the idea of having Kerberos authentication supported within GIRAPH. Hopefully it would use our fast GIRAPH-37 IPC, but could also interoperate with Hadoop security.

Issue Links

• blocks: GIRAPH-200 remove hadoop RPC and keep just netty
• blocks: GIRAPH-282 Make netty the default RPC transport
• is depended upon by: GIRAPH-852 Add SASL unit tests
• is related to: GIRAPH-363 Fix hadoop_0.23 profile broken by GIRAPH-211
• is related to: GIRAPH-583 Problem with authentication on Hadoop 0.23
• is related to: GIRAPH-265 Enable Zookeeper security support within Giraph
• is related to: GIRAPH-200 remove hadoop RPC and keep just netty
• is related to: GIRAPH-262 Netty optimization to handle requests locally whenever possible
• relates to: GIRAPH-212 Security is busted since GIRAPH-168
• relates to: GIRAPH-37 Implement Netty-backed IPC
• relates to: AVRO-405 Netty-based Java RPC server
• relates to: AVRO-641 add SASL to socket transport
• relates to: HADOOP-4487 Security features for Hadoop
• relates to: HADOOP-6419 Change RPC layer to support SASL based mutual authentication
• relates to: ZOOKEEPER-823 update ZooKeeper java client to optionally use Netty for connections
• relates to: GIRAPH-362 Address master task id for communication for master (known issue from GIRAPH-211)
• relates to: GIRAPH-262 Netty optimization to handle requests locally whenever possible
• relates to: GIRAPH-289 Add thread and channel pooling to NettyClient and NettyServer
• relates to: HADOOP-8078 Add capability to turn on security in unit tests.

Sub-Tasks

• Keep track of the task id in ChannelRotater to send requests without knowing the worker id upfront: Resolved