Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2898

Kafka source test will freeze on JDK8

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After committing FLUME-2821, I've noticed that the test case doesn't work properly on JDK8. It gets stuck:

      "main" #1 prio=5 os_prio=31 tid=0x00007fa662000800 nid=0x1703 runnable [0x0000700000217000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
      	at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
      	at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
      	- locked <0x000000076bc0dd78> (a sun.nio.ch.Util$2)
      	- locked <0x000000076bc0dd68> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x000000076bc0dc48> (a sun.nio.ch.KQueueSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
      	at org.apache.kafka.common.network.Selector.select(Selector.java:425)
      	at org.apache.kafka.common.network.Selector.poll(Selector.java:254)
      	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256)
      	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
      	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
      	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
      	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.awaitMetadataUpdate(ConsumerNetworkClient.java:134)
      	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorKnown(AbstractCoordinator.java:184)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:886)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:853)
      	at org.apache.flume.source.kafka.KafkaSource.doStart(KafkaSource.java:352)
      	at org.apache.flume.source.BasicSourceSemantics.start(BasicSourceSemantics.java:83)
      	- locked <0x00000006c81d1698> (a org.apache.flume.source.kafka.KafkaSource)
      	at org.apache.flume.source.kafka.TestKafkaSource.testOffsets(TestKafkaSource.java:97)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
      

      I think that it's caused by different behavior of InetAddress.getLocalHost().getHostAddress in the KafkaSourceEmbeddedKafka when running on JDK8. When I've set the value of HOST explicitly to localhost, everything worked.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jarcec Jarek Jarcec Cecho
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: