Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6569

Reflection in OffsetIndex and TimeIndex construction

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.3.0
    • core
    • None

    Description

      kafka.log.AbstractIndex uses the Logging mixin to lazily initialize loggers for any concrete type that inherits from it. This works great, except that the LazyLogging trait uses reflection to compute the logger name.

      When you have hundreds of thousands of log segments to load on startup the extra cost adds up.

      I've attached flame graphs from broker startup on a broker that has about 12TB of log segments to load, and a second flame graph after changing AbstractIndex to statically initialize a logger.

      Attachments

        1. after.png
          420 kB
          Kyle Ambroff-Kao
        2. before.png
          532 kB
          Kyle Ambroff-Kao

        Issue Links

          Activity

            People

              ambroff Kyle Ambroff-Kao
              ambroff Kyle Ambroff-Kao
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: