Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2426

First tuples fail after worker is respawn

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.2
    • 2.0.0, 1.2.0, 1.1.2
    • storm-kafka-client
    • None

    Description

      Topology with two Kafka spouts (org.apache.storm.kafka.spout.KafkaSpout) reading from two different topics with same consumer group ID.

      1. Kill the only worker process for topology
      2. Storm creates new worker
      3. Kafka starts rebalancing (log line 15-16)
      4. Kafka rebalancing done (log line 18-19)
      5. Kafka topics read and tuples emitted (log line 28-29)
      6. Tuples immediately fail (log line 30-33)

      The delay between tuples emitted and tuples failing is just some 10 ms. No bolts in topology received the tuples.

      What could cause this? The assumption is that there are uncommitted messages in Spout when it is killed and those are retried.

      Attachments

        1. 2017-03-20-Kafka-spout-issue.txt
          9 kB
          Antti Järvinen
        2. 2017-03-21-Timeout-ticks.txt
          1 kB
          Antti Järvinen

        Issue Links

          Activity

            People

              srdo Stig Rohde Døssing
              EitZei Antti Järvinen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: