[FLINK-7597] broken flink-connectors-kinesis setup in Intellij that potentially results from improper pom.xml

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.3.2, 1.4.0
• Component/s: Connectors / Kinesis
• Assignee: Bowen Li
• Reporter: Bowen Li
• Created: 07/Sep/17 06:29
• Updated: 07/Sep/17 21:14
• Resolved: 07/Sep/17 21:14

Description

I use Intellij to develop flink and flink-connectors-kinesis. I imported the whole flink src code into Intellij, and Intellij treats flink-connectors-kinesis as a module. The project structure in intellij looks like this: https://imgur.com/a/uK3Fd

Here's the problem: The flink-connectors-kinesis module always complains about not being able to find dependencies like amazon-kinesis-producer, amazon-kinesis-client, flink-streaming-java_2.11, etc. Seems like Intellij cannot properly parse /flink-connectors-kinesis/pom.xml. And Intellij always suggest I add those dependencies to flink-connectors/pom.xml. In short, flink-connectors-kinesis won't compile in my Intellij until I added those dependencies to flink-connectors/pom.xml.

My flink-connectors/pom.xml file ends up like this all the time:

C02SD32LG8WP:flink-connectors Bowen$ git diff
diff --git a/flink-connectors/pom.xml b/flink-connectors/pom.xml
index bc3f82f..2b001f5 100644
--- a/flink-connectors/pom.xml
+++ b/flink-connectors/pom.xml
@@ -71,6 +71,16 @@ under the License.
                        <artifactId>jsr305</artifactId>
                        <scope>provided</scope>
                </dependency>
+               <dependency>
+                       <groupId>com.amazonaws</groupId>
+                       <artifactId>amazon-kinesis-producer</artifactId>
+                       <version>0.12.5</version>
+               </dependency>
+               <dependency>
+                       <groupId>com.amazonaws</groupId>
+                       <artifactId>amazon-kinesis-client</artifactId>
+                       <version>1.8.1</version>
+               </dependency>
+               <dependency>
+                       <groupId>org.apache.flink</groupId>
+                       <artifactId>flink-streaming-java_2.11</artifactId>
+                       <version>1.4-SNAPSHOT</version>
+               </dependency>
        </dependencies>

        <!-- See main pom.xml for explanation of profiles -->

FYI, building flink-connectors-kinesis from command line always works.

tzulitai Do you use Intellij? If so, how do you properly set up the flink-connectors-kinesis project in Intellij to be able to retrieve dependencies?