Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-495

Add delayed retries to KafkaSpout

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.9.3
    • 0.10.0
    • storm-kafka
    • all environments

    Description

      If a tuple in the topology originates from the KafkaSpout from the external/storm-kafka sources, and if a bolt in the topology indicates a failure by calling fail() on its OutputCollector, the KafkaSpout will immediately retry the message.

      We wish to use this failure and retry behavior in our ingestion system whenever we experience a recoverable error from a downstream system, such as a 500 or 503 error from a service we depend on. But with the current KafkaSpout behavior, doing so results in a tight loop where we retry several times over a few seconds and then give up. I want to be able to delay retry to give the downstream service some time to recover. Ideally, I would like to have configurable, exponential backoff retry.

      Attachments

        Activity

          People

            rkilgore Rick Kilgore
            rkilgore Rick Kilgore
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: