Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1440

Kylin streaming stop refreshed when one replica of partition goes down

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: None
    • Component/s: NRT Streaming
    • Labels:
      None

      Description

      java.lang.NullPointerException: unable to find leadBroker with config:org.apache.kylin.streaming.KafkaClusterConfig@3c1 partitionId:0
      at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:208)
      at org.apache.kylin.streaming.StreamingUtil.getFirstAndLastOffset(StreamingUtil.java:99)
      at org.apache.kylin.streaming.StreamingUtil.findClosestOffsetWithDataTimestamp(StreamingUtil.java:88)
      at org.apache.kylin.job.streaming.StreamingBootstrap$4.run(StreamingBootstrap.java:287)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      [root@kylin-dev7-eaz-704018 kafka_2.10-0.8.2.0]# bin/kafka-topics.sh --describe --zookeeper
      nous.seo_session
      Topic:nous.seo_session PartitionCount:3 ReplicationFactor:3Configs:retention.ms=604800000
      Topic: nous.seo_session Partition: 0 Leader: 8Replicas: 1,8,9 Isr: 8,9
      Topic: nous.seo_session Partition: 1 Leader: 2Replicas: 2,9,10 Isr: 2,10,9
      Topic: nous.seo_session Partition: 2 Leader: 4Replicas: 4,10,11 Isr: 4,10,11

      It should be a normal situation and Kafka can still work normally. However Something in Kylin streaming went wrong when dealing with such cases.

        Attachments

          Activity

            People

            • Assignee:
              yaho Zhong Yanghong
              Reporter:
              qianqiaoneng qianqiaoneng
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: