[AVRO-405] Netty-based Java RPC server

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: java
• Hadoop Flags: Reviewed
• Assignee: harry wang
• Reporter: Todd Lipcon
• Created: 05/Feb/10 22:45
• Updated: 02/Jul/12 00:11
• Resolved: 01/Jul/10 21:26

Description

A nonblocking RPC server based on Netty should be more scalable than the current implementation.

We should provide two mechanisms for interfacing the RPC server to the implementations:

1) "Blocking" RPC implementations run inside a worker threadpool. Implementators would not know that they're working in a non-blocking context.
2) "Event-driven" RPC implementations that receive requests and some kind of request context. They are responsible for eventually calling context.respond(response) or somesuch. This would allow more scalable interaction with downstream services.

I propose we focus on (1) first.

Issue Links

• is related to: GIRAPH-211 Add secure authentication to Netty IPC
• relates to: AVRO-406 Support streaming RPC calls
• relates to: AVRO-539 Allow asynchronous clients to specify a callback to be run when server processing completes
• relates to: AVRO-585 add async RPC cllient
• relates to: HDFS-916 Rewrite DFSOutputStream to use a single thread with NIO
• relates to: ZOOKEEPER-733 use netty to handle client connections
• relates to: HBASE-2400 new connector for Avro RPC access to HBase cluster
• relates to: HBASE-2182 rpc/ipc refactor; x-version compatibility, nio, async, enveloping, sane timeouts, etc.