Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1374

LogCleaner (compaction) does not support compressed topics

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.9.0.0
    • None

    Description

      This is a known issue, but opening a ticket to track.

      If you try to compact a topic that has compressed messages you will run into
      various exceptions - typically because during iteration we advance the
      position based on the decompressed size of the message. I have a bunch of
      stack traces, but it should be straightforward to reproduce.

      Attachments

        1. KAFKA-1374_2014-08-09_16:18:55.patch
          12 kB
          Manikumar
        2. KAFKA-1374_2014-08-12_22:23:06.patch
          13 kB
          Manikumar
        3. KAFKA-1374_2014-09-23_21:47:12.patch
          17 kB
          Manikumar
        4. KAFKA-1374_2014-10-03_18:49:16.patch
          17 kB
          Manikumar
        5. KAFKA-1374_2014-10-03_19:17:17.patch
          17 kB
          Manikumar
        6. KAFKA-1374_2015-01-18_00:19:21.patch
          18 kB
          Manikumar
        7. KAFKA-1374_2015-05-18_22:55:48.patch
          20 kB
          Manikumar
        8. KAFKA-1374_2015-05-19_17:20:44.patch
          40 kB
          Joel Jacob Koshy
        9. KAFKA-1374.patch
          20 kB
          Joel Jacob Koshy
        10. KAFKA-1374.patch
          10 kB
          Manikumar

        Issue Links

          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1944 Rename LogCleaner and related classes to LogCompactor

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-388 Log compaction on checkpoint topics fails with compression

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-398 Enable compression in Kafka checkpoint and changelog topics

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SAMZA-586 LoggedStore can send compressed messages to log-compacted Kafka topic

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. KAFKA-1499 Broker-side compression configuration

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-1581 Log cleaner should have an option to ignore messages without keys

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              omkreddy Manikumar
              jjkoshy Joel Jacob Koshy
              Guozhang Wang Guozhang Wang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: