Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6939

Change the default of log.message.timestamp.difference.max.ms to 500 years

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: log
    • Labels:
      None

      Description

      If producer incorrectly provides timestamp in microsecond (not in millisecond), the record is accepted by default and can cause broker to roll the segment files continuously. And on a heavily used broker, this will generate a lot of index files, which then causes the broker to hit `vm.max_map_count`.

      So I'd like to suggest changing the default for log.message.timestamp.difference.max.ms to 15768000000000 (500 years * 365 days * 86400 seconds * 1000). This would reject timestamp in microsecond from producer and still allow most historical data to be stored in Kafka.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              badai Badai Aqrandista
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: