Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.7.0
    • Component/s: Test
    • Labels:
      None

      Description

      KafkaSource tests are flaky, and tend to fail. I investigated this issue and discovered the following issues:

      • KafkaSource hangs in doStart at line 413:
        KafkaSourceEmbeddedKafka.java
         it = consumer.poll(1000).iterator();
        

      I fixed that by simply changing HOST in KafkaSourceEmbeddedKafka to localhost.

      • Out of memory exception. I fixed that by properly configuring MAVEN_OPTS, as stated in the documentation.

        Activity

        Hide
        mpercy Mike Percy added a comment -

        +1. This works for me.

        Show
        mpercy Mike Percy added a comment - +1. This works for me.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 209c1e2548080525919fe78a1a61cc8fc7ab306f in flume's branch refs/heads/trunk from Mike Percy
        [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=209c1e2 ]

        FLUME-2936. KafkaSource tests: Use loopback instead of host address

        TestKafkaSource is flaky because it tries to run Kafka on the host
        address instead of the loopback address. This patch fixes that problem.

        It also introduces a bug fix related to passing properties to
        KafkaSourceEmbeddedKafka.

        (Lior Zeno via Mike Percy)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 209c1e2548080525919fe78a1a61cc8fc7ab306f in flume's branch refs/heads/trunk from Mike Percy [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=209c1e2 ] FLUME-2936 . KafkaSource tests: Use loopback instead of host address TestKafkaSource is flaky because it tries to run Kafka on the host address instead of the loopback address. This patch fixes that problem. It also introduces a bug fix related to passing properties to KafkaSourceEmbeddedKafka. (Lior Zeno via Mike Percy)
        Hide
        mpercy Mike Percy added a comment -

        I just pushed this to trunk. Thanks very much for the fix, Lior!

        Show
        mpercy Mike Percy added a comment - I just pushed this to trunk. Thanks very much for the fix, Lior!
        Hide
        hudson Hudson added a comment -

        UNSTABLE: Integrated in Flume-trunk-hbase-1 #175 (See https://builds.apache.org/job/Flume-trunk-hbase-1/175/)
        FLUME-2936. KafkaSource tests: Use loopback instead of host address (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=209c1e2548080525919fe78a1a61cc8fc7ab306f)

        • flume-ng-sources/flume-kafka-source/src/test/java/org/apache/flume/source/kafka/KafkaSourceEmbeddedKafka.java
        Show
        hudson Hudson added a comment - UNSTABLE: Integrated in Flume-trunk-hbase-1 #175 (See https://builds.apache.org/job/Flume-trunk-hbase-1/175/ ) FLUME-2936 . KafkaSource tests: Use loopback instead of host address (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=209c1e2548080525919fe78a1a61cc8fc7ab306f ) flume-ng-sources/flume-kafka-source/src/test/java/org/apache/flume/source/kafka/KafkaSourceEmbeddedKafka.java

          People

          • Assignee:
            liorze Lior Zeno
            Reporter:
            liorze Lior Zeno
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development