Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2882

Add constructor cache for Snappy and LZ4 Output/Input stream in Compressor.java

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.2.1
    • Fix Version/s: 0.9.0.1
    • Component/s: consumer, producer
    • Labels:
      None

      Description

      In `wrapForOutput` and `wrapForInput` methods of `org.apache.kafka.common.record.Compressor`, `Class.forName("[compression codec]")` and `getConstructor` methods are invoked for each `wrapForOutput` / `wrapForInput` call. Reflection calls are expensive and impact performance at high volumes. This patch adds a cache for `Constructor` to reduce the reflection overhead.

      In our production deployments, this has reduced producer CPU usage by about 20%

        Attachments

          Activity

            People

            • Assignee:
              logarithm Maksim Logvinenko
              Reporter:
              logarithm Maksim Logvinenko
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: