[KAFKA-4011] allow sizing RequestQueue in bytes

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.10.0.0
• Component/s: core
• Assignee: Unassigned
• Reporter: Radai Rosenblatt
• Created: 01/Aug/16 21:56
• Updated: 14/Nov/16 21:50
• Resolved: 14/Nov/16 21:50

Description

currently RequestChannel's requestQueue is sized in number of requests:

RequestChannel.scala

private val requestQueue = new ArrayBlockingQueue[RequestChannel.Request](queueSize)

under the assumption that the end goal is a bound on server memory consumption, this requires the admin to know the avg request size.

I would like to propose sizing the requestQueue not by number of requests, but by their accumulated size (Request.buffer.capacity). this would probably make configuring and sizing an instance easier.

there would need to be a new configuration setting for this (queued.max.bytes?) - which could be either in addition to or instead of the current queued.max.requests setting

Issue Links

• links to: GitHub Pull Request #1714