[FLINK-22610] The test-jar and test-source-jar of flink-connector-kafka should include all classes

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.13.0, 1.12.3
• Component/s: Connectors / Kafka
• Assignee: Unassigned
• Reporter: LIU Xiao
• Created: 09/May/21 04:27
• Updated: 25/Jun/21 07:58

Description

The test-jar of old kafka connector (flink-connector-kafka-base and flink-connector-kafka-0.11) includes convenient utility classes (KafkaTestEnvironment and KafkaTestEnvironmentImpl, etc.) to start an embedded kafka in unit test, and we used the utility classes to build some test cases for our project.

Now the utility classes other than KafkaTestEnvironmentImpl seem to be gone in test-jar of new kafka connector (flink-connector-kafka), and I find that is because they are not included in the configuration of maven-jar-plugin in pom.xml:

...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>test-jar</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/KafkaTestEnvironmentImpl*</include>
                    <include>META-INF/LICENSE</include>
                    <include>META-INF/NOTICE</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>
...

This configuration seems to be inherited from flink-connector-kafka-0.11, but actually the configuration of flink-connector-kafka-base should be used:

...
<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>test-jar</goal>
            </goals>
        </execution>
    </executions>
</plugin>
...

The test-source-jar has similar problem:

...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-source-plugin</artifactId>
    <executions>
        <execution>
            <id>attach-test-sources</id>
            <goals>
                <goal>test-jar-no-fork</goal>
            </goals>
            <configuration>
                <archive>
                    <!-- Globally exclude maven metadata, because it may accidentally bundle files we don't intend to -->
                    <addMavenDescriptor>false</addMavenDescriptor>
                </archive>
                <includes>
                    <include>**/KafkaTestEnvironmentImpl*</include>
                    <include>META-INF/LICENSE</include>
                    <include>META-INF/NOTICE</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>
...

I think it should be:

...
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-source-plugin</artifactId>
    <executions>
        <execution>
            <id>attach-test-sources</id>
            <goals>
                <goal>test-jar-no-fork</goal>
            </goals>
            <configuration>
                <archive>
                    <!-- Globally exclude maven metadata, because it may accidentally bundle files we don't intend to -->
                    <addMavenDescriptor>false</addMavenDescriptor>
                </archive>
            </configuration>
        </execution>
    </executions>
</plugin>
...