[KAFKA-4293] ByteBufferMessageSet.deepIterator burns CPU catching EOFExceptions - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 0.10.0.1
Fix Version/s: None
Component/s: core
Labels:
None

Description

around line 110:

try {
    while (true)
        innerMessageAndOffsets.add(readMessageFromStream(compressed))
} catch {
    case eofe: EOFException =>
    // we don't do anything at all here, because the finally
    // will close the compressed input stream, and we simply
    // want to return the innerMessageAndOffsets

the only indication the code has that the end of the oteration was reached is by catching EOFException (which will be thrown inside readMessageFromStream()).

profiling runs performed at linkedIn show 10% of the total broker CPU time taken up by Throwable.fillInStack() because of this behaviour.

unfortunately InputStream.available() cannot be relied upon (concrete example - GZipInputStream will not correctly return 0) so the fix would probably be a wire format change to also encode the number of messages.