Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
0.8.1
-
None
-
None
Description
This is part of the investigation into slow shutdowns in 0.8.1. While
logging contributes to bulk of the regression, this one also adds
quite a bit of overhead:
In addLeaderAndIsrRequest (called for every partition that is led by the
broker being shut down) we also add an UpdateMetadataRequest - each call to
addUpdateMetadataRequests does two traversals over all (global)
partitions. I think it should be straightforward to optimize this a bit.
Marking as critical, since it is not as big an overhead as the logging.
Attachments
Attachments
Issue Links
- is depended upon by
-
KAFKA-1380 0.8.1.1 release candidate
- Closed
BTW, here are the configs and steps I used for this,
KAFKA-1342andKAFKA-1350:Four brokers, 100 topics, eight partitions each.
Log4j and server configs by broker:
https://gist.github.com/anonymous/9906088
https://gist.github.com/anonymous/9906092
https://gist.github.com/anonymous/9906096
https://gist.github.com/anonymous/9906102
https://gist.github.com/anonymous/9906144
https://gist.github.com/anonymous/9906148
https://gist.github.com/anonymous/9906153
https://gist.github.com/anonymous/9906157
Producer performance: https://gist.github.com/anonymous/9906163
(At the end, just grep in the controller's request log and extract local time)
grep ControlledShutdownRequest logs/kafka-request*