Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2178

Loss of highwatermarks on incorrect cluster shutdown/restart

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 0.8.2.1
    • None
    • replication
    • None
    • Patch

    Description

      ReplicaManager flushes highwatermarks only for partitions which it recieved from Controller.
      If Controller sends incomplete list of partitions then ReplicaManager will write incomplete list of highwatermarks.
      As a result one can lose a lot of data during incorrect broker restart.

      We got this situation in real life on our cluster.

      Attachments

        1. KAFKA-2178.patch
          4 kB
          Alexey Ozeritskiy

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-3693 Lost highwatermark at broker start-up

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              aozeritsky Alexey Ozeritskiy
              Jun Rao Jun Rao
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: