Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9992

EmbeddedKafkaCluster.deleteTopicAndWait not working with kafka_2.13

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.1
    • Fix Version/s: 2.6.0, 2.4.2, 2.5.1
    • Component/s: packaging, streams
    • Labels:
      None

      Description

      Kafka Streams artifact is depending on kafka_2.12 as of now, it is in the kafka-streams-2.4.1.pom:

      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.12</artifactId>
        <version>2.4.1</version>
        <scope>test</scope>
      </dependency>  
      

      But it is not hardcoded, whatever scala version was used to compile this component before uploading, that will be present in the pom.

      When I'm using these deps:

      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-streams</artifactId>
        <version>2.4.1</version>
        <classifier>test</classifier>
        <scope>test</scope>
      </dependency>
      
      <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.13</artifactId>
        <version>2.4.1</version>
        <classifier>test</classifier>
        <scope>test</scope>
      </dependency>
      

      My test fails with the following exception (deleteTopicAndWait is called in my @After method):

      java.lang.NoSuchMethodError: scala.collection.JavaConverters.setAsJavaSetConverter(Lscala/collection/Set;)Lscala/collection/convert/Decorators$AsJava;
              at org.apache.kafka.streams.integration.utils.EmbeddedKafkaCluster$TopicsDeletedCondition.conditionMet(EmbeddedKafkaCluster.java:316)
              at org.apache.kafka.test.TestUtils.lambda$waitForCondition$4(TestUtils.java:370)
              at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:417)
              at org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:385)
              at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:368)
              at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:356)
              at org.apache.kafka.streams.integration.utils.EmbeddedKafkaCluster.deleteTopicsAndWait(EmbeddedKafkaCluster.java:266)
              at org.apache.kafka.streams.integration.utils.EmbeddedKafkaCluster.deleteTopicAndWait(EmbeddedKafkaCluster.java:221)
      

      I modified kafka build locally to separate artifacts based on scala version just like it is done with kafka core, and I pulled in kafka-streams_2.13 from my local mvn repo and test was working again.

      I was only trying with 2.4.1, but I'm assuming other versions are also affected, please add the proper versions and proper components too (in case it's not packaging).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                akatona Andras Katona
                Reporter:
                akatona Andras Katona
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: