Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-257

fix ConcurrentModificationException in KafkaIO

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0-incubating
    • Component/s: io-java-kafka
    • Labels:
      None

      Description

      Thomas Groh reported the following exception with KafkaIO while using with DirectRunner. Fixing it in https://github.com/apache/incubator-beam/pull/290

       java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access
      	at org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1324)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1255)
      	at com.google.common.io.Closeables.close(Closeables.java:79)
      	at org.apache.beam.sdk.io.kafka.KafkaIO$UnboundedKafkaReader.close(KafkaIO.java:1050)
      	at org.apache.beam.runners.direct.UnboundedReadEvaluatorFactory$UnboundedReadEvaluator.finishBundle(UnboundedReadEvaluatorFactory.java:167)
      	at org.apache.beam.runners.direct.TransformExecutor.finishBundle(TransformExecutor.java:161)
      	at org.apache.beam.runners.direct.TransformExecutor.call(TransformExecutor.java:116)
      	at org.apache.beam.runners.direct.TransformExecutor.call(TransformExecutor.java:41)
      	at java.util.concurrent.FutureTask.run(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.lang.Thread.run(Unknown Source)
      

        Attachments

          Activity

            People

            • Assignee:
              rangadi Raghu Angadi
              Reporter:
              rangadi Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: