Avoid a buffer copy while replying to RPC requests.

• Key: HADOOP-4813
• Type: Improvement
• Status: Open
• Priority: Major
• Assignee: Unassigned
• Reporter: Raghu Angadi
• Component/s: ipc

Description

RPC server first serializes RPC response to a ByteArrayOutputStream and then creates a new array to write to socket. For most responses the RPC handler is able to write the entire response in-line. If we could use the same buffer used by ByteArrayOutputStream, we can avoid this copy.

As mentioned in HADOOP-4802, yet another copy could be avoided (in most cases) if we use a static direct buffer for the responses (not proposed for this jira).

Comments

[Doug Cutting]

As a related matter, I'd like to change the underlying IPC layer to take, instead of a Writable a List<ByteBuffer> for requests and return a ByteBuffer for responses. Then we can layer an RPC implementation on top that does not copy buffers but rather assembles a list of them. The simplest implementation would allocate new buffers per request, then let them get GC'd. Such request and response buffers should be short-lived and cheap to GC, but if that proves to be a problem we could consider pooling them.

[Raghu Angadi]

Right. It may not require changes to how objects are read and written in IPC. In stead of ByteArrayStreams we just need to use our own input and output streams that use a list of buffers instead of single buffer. We need to propagate this list through the write code path on Server and client.

The way I see it the bytes on wire don't actually change. Is that approximately what you are proposing?

[Doug Cutting]

> bytes on wire don't actually change. Is that approximately what you are proposing?

Yes. We might add an output operation like, writeBuffer(ByteBuffer) that adds the stream's current output buffer to the list if anything has been buffered, then adds the passed buffer, and resets the stream's buffer