Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3044

KafkaSink should avoid to call method without timeout param

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 1.7.0
    • None
    • Sinks+Sources
    • None

    Description

      There are several method call in kafka sink with no timeout params, in some cases, kafka sink will await forever if no interruption.
      For example´╝î
      case 1 : When we stop kafka sink, it will call producer.close() without timeout param. This will cause another close method in producer with timeout param as (Long.MAX_VALUE, TimeUnit.MILLISECONDS). Then it will wait producer's IOThread exited, if the IOThread can't exit normally, kafka sink will block all the time and there is no effect of kill -TERM.
      case 2 : When an uncaught exception happened such as Correlation ID does not match in Kafka Producer Sender thread, the sender will lost this request. If we call producer.flush(), there is a await() without timeout param, kafka sink will block all the time except restart it.
      case 3 : Like case 2, when we call future.get() without timeout param but the request in Kafka Producer lost, kafka sink will also block all the time.

      Attachments

        1. FLUME-3044.patch
          3 kB
          dengkai

        Activity

          People

            lolee_k dengkai
            lolee_k dengkai
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: