Flume
  1. Flume
  2. FLUME-2484

NullPointerException in Kafka Sink test

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Not A Bug
    • Affects Version/s: v1.6.0
    • Fix Version/s: v1.7.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      Kafka Sink test fails on Travis with NullPointerException:

      https://travis-ci.org/Stratio/flume/jobs/36814710#L6560

      Running org.apache.flume.sink.kafka.TestKafkaSink
      Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 17.061 sec <<< FAILURE!
      testStaticTopic(org.apache.flume.sink.kafka.TestKafkaSink) Time elapsed: 1823 sec <<< ERROR!
      java.lang.NullPointerException
      at org.apache.flume.sink.kafka.TestKafkaSink.testStaticTopic(TestKafkaSink.java:113)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
      at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
      at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
      at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
      Results :
      Tests in error:
      testStaticTopic(org.apache.flume.sink.kafka.TestKafkaSink)

        Activity

        Hide
        Santiago M. Mola added a comment -

        Gwen Shapira Maybe you have some hint for a fix?

        Show
        Santiago M. Mola added a comment - Gwen Shapira Maybe you have some hint for a fix?
        Hide
        Santiago M. Mola added a comment -

        I'm still getting this error with latest code: https://travis-ci.org/Stratio/flume/jobs/38727640#L8030

        Show
        Santiago M. Mola added a comment - I'm still getting this error with latest code: https://travis-ci.org/Stratio/flume/jobs/38727640#L8030
        Hide
        Gwen Shapira added a comment -

        I can't reproduce the issue or figure out what went wrong just by looking at the test.

        I take it that other KafkaSink tests are successful?
        Can you attach logs or console output from the test execution? I need more information on what's going on there.

        If you are trying to debug yourself, this is the line thats failing:
        String fetchedMsg = new String((byte[]) testUtil.getNextMessageFromConsumer(
        TestConstants.STATIC_TOPIC).message())

        testUtil.getNextMessageFromConsumer can return null if there was nothing to consume within 1s. Looking at the time the test took, I suspect this is the issue.
        Which probably means kafkaSink.process() in "prepareAndSend" failed without returning a BACKOFF status.

        Logs or any other extra insights on the failure will be appreciated. I'd be more helpful, but I can't get this to reproduce.

        Show
        Gwen Shapira added a comment - I can't reproduce the issue or figure out what went wrong just by looking at the test. I take it that other KafkaSink tests are successful? Can you attach logs or console output from the test execution? I need more information on what's going on there. If you are trying to debug yourself, this is the line thats failing: String fetchedMsg = new String((byte[]) testUtil.getNextMessageFromConsumer( TestConstants.STATIC_TOPIC).message()) testUtil.getNextMessageFromConsumer can return null if there was nothing to consume within 1s. Looking at the time the test took, I suspect this is the issue. Which probably means kafkaSink.process() in "prepareAndSend" failed without returning a BACKOFF status. Logs or any other extra insights on the failure will be appreciated. I'd be more helpful, but I can't get this to reproduce.
        Hide
        simonchi added a comment -
        Show
        simonchi added a comment - here is my kafka sink https://github.com/cwtree/flume-kafka-sink
        Hide
        Bessenyei Balázs Donát added a comment -

        Tests in Kafka sink seem to be working now.

        Can we mark this ticket as resolved (Not a bug / Not a problem)?

        Show
        Bessenyei Balázs Donát added a comment - Tests in Kafka sink seem to be working now. Can we mark this ticket as resolved (Not a bug / Not a problem)?
        Hide
        Lior Zeno added a comment -

        Sure, go ahead.

        Show
        Lior Zeno added a comment - Sure, go ahead.

          People

          • Assignee:
            Unassigned
            Reporter:
            Santiago M. Mola
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development