Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6514

Cannot start Flink Cluster in standalone mode

    Details

      Description

      The changes introduced for FLINK-5998 change what goes into the flink-dost fat jar. As it is, this means that trying to start a cluster results in a ClassNotFoundException of LogFactory in commons-logging.

      One solution is to now make the shaded Hadoop jar a proper fat-jar, so that we again have all the dependencies.

        Issue Links

          Activity

          Hide
          aljoscha Aljoscha Krettek added a comment -

          Proposed solution (that has other possible complications): https://github.com/apache/flink/pull/3856

          Show
          aljoscha Aljoscha Krettek added a comment - Proposed solution (that has other possible complications): https://github.com/apache/flink/pull/3856
          Hide
          greghogan Greg Hogan added a comment -

          Aljoscha Krettek are you running a newer version of Maven? I was seeing this issue yesterday until reverting to 3.2.5.

          Show
          greghogan Greg Hogan added a comment - Aljoscha Krettek are you running a newer version of Maven? I was seeing this issue yesterday until reverting to 3.2.5.
          Hide
          aljoscha Aljoscha Krettek added a comment -

          Checking this out now.

          Show
          aljoscha Aljoscha Krettek added a comment - Checking this out now.
          Hide
          aljoscha Aljoscha Krettek added a comment -

          Yes, this seems to have been caused by my Maven version.

          Show
          aljoscha Aljoscha Krettek added a comment - Yes, this seems to have been caused by my Maven version.
          Hide
          Zentol Chesnay Schepler added a comment -

          Why was this closed as not a problem? Having to downgrade maven is not a good solution, especially when we don't even know what the actual underlying problem is.

          Show
          Zentol Chesnay Schepler added a comment - Why was this closed as not a problem? Having to downgrade maven is not a good solution, especially when we don't even know what the actual underlying problem is.
          Hide
          aljoscha Aljoscha Krettek added a comment -

          Chesnay Schepler You're right, re-opening.

          Show
          aljoscha Aljoscha Krettek added a comment - Chesnay Schepler You're right, re-opening.
          Hide
          rmetzger Robert Metzger added a comment -

          We could use the maven enforcer plugin to enforce maven version 3.2.5 in the release profile.
          But I think almost everybody doing a custom Flink build will run into the issue.

          Show
          rmetzger Robert Metzger added a comment - We could use the maven enforcer plugin to enforce maven version 3.2.5 in the release profile. But I think almost everybody doing a custom Flink build will run into the issue.
          Hide
          StephanEwen Stephan Ewen added a comment -

          There are more issues. The way this is set up right now also drops transitive dependencies from Flink that should not be dropped (compression, parts of Avro support, ...)

          I have a more comprehensive fix for this...

          Show
          StephanEwen Stephan Ewen added a comment - There are more issues. The way this is set up right now also drops transitive dependencies from Flink that should not be dropped (compression, parts of Avro support, ...) I have a more comprehensive fix for this...
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user StephanEwen opened a pull request:

          https://github.com/apache/flink/pull/3876

          FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly

          This fixes the issue that Flink cannot be started locally if built with Maven 3.3+

          There are two big fixes in this pull request, because they do not build/pass tests individually. The wrong Mesos dependencies where the reason that the broken Hadoop fat jar building actually passed the Yarn tests.

          1. Hadoop Uber Jar
          • This builds a proper Hadoop Uber Jar with all of Hadoop's needed dependencies. The prior build was missing many important dependencies in the Hadoop Uber Jar.
          • The Hadoop-jar is no longer excluded in `flink-dist` via setting the dependency to `provided`, but by explicit exclusion. That way, Hadoop's transitive dependencies are not excluded from other dependencies as well. Before this patch, various decompression and Avro were broken in a Flink build, due to accidental exclusion of their transitive dependencies.
          1. Dependency fixing
          • This also fixes the dependencies of `flink-mesos`, which made all of Hadoop's transitive dependencies its own dependencies, by promoting them during shading. That way, Flink had various unnecessary dependencies in its `flink-dist` jar.
          • Incidentally, that brought Hadoop's accidentally excluded dependencies back in, but into the `flink-dist` jar, not the `shaded-hadoop2` jar.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/StephanEwen/incubator-flink fix_fat_jar

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3876.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3876


          commit 86803d23dfdc56f2be64274a6a90a76c1e782f08
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T13:12:50Z

          FLINK-6546 [build] Fix dependencies of flink-mesos

          • This makes all flink-related dependencies 'provided' to not have the
            transitive dependencies promoted
          • Drops the unnecessary dependency on the Hadoop artifact
          • Adds directly referenced libraries, like jackson
          • Deactivates default logging of tests

          commit a14137c109e73738a3d1f89a3d99e2fd2a799219
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:32:03Z

          [build] Reduce flink-avro's compile dependency from 'flink-java' to 'flink-core'

          commit 32e8574498d7963e2ab58f1530b41a6853f23601
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:35:43Z

          FLINK-6514 [build] Remove 'flink-shaded-hadoop2' from 'flink-dist' via exclusions

          This is more tedious/manual than setting it to 'provided' once, but it
          is also correct.

          For example, in the case of Hadoop 2.3, having 'flink-shaded-hadoop2' as 'provided'
          removes other needed dependencies as well, such as 'org.codehaus.jackson' from avro.

          commit efaed902a78c6a7f236e0dad4f72ed7ae8bad1c0
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T11:52:25Z

          FLINK-6514 [build] Merge bin and lib assembly

          commit df8efd5cdadaeef9323472f20871871c94d14af5
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:00:03Z

          FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user StephanEwen opened a pull request: https://github.com/apache/flink/pull/3876 FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly This fixes the issue that Flink cannot be started locally if built with Maven 3.3+ There are two big fixes in this pull request, because they do not build/pass tests individually. The wrong Mesos dependencies where the reason that the broken Hadoop fat jar building actually passed the Yarn tests. Hadoop Uber Jar This builds a proper Hadoop Uber Jar with all of Hadoop's needed dependencies. The prior build was missing many important dependencies in the Hadoop Uber Jar. The Hadoop-jar is no longer excluded in `flink-dist` via setting the dependency to `provided`, but by explicit exclusion. That way, Hadoop's transitive dependencies are not excluded from other dependencies as well. Before this patch, various decompression and Avro were broken in a Flink build, due to accidental exclusion of their transitive dependencies. Dependency fixing This also fixes the dependencies of `flink-mesos`, which made all of Hadoop's transitive dependencies its own dependencies, by promoting them during shading. That way, Flink had various unnecessary dependencies in its `flink-dist` jar. Incidentally, that brought Hadoop's accidentally excluded dependencies back in, but into the `flink-dist` jar, not the `shaded-hadoop2` jar. You can merge this pull request into a Git repository by running: $ git pull https://github.com/StephanEwen/incubator-flink fix_fat_jar Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3876.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3876 commit 86803d23dfdc56f2be64274a6a90a76c1e782f08 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T13:12:50Z FLINK-6546 [build] Fix dependencies of flink-mesos This makes all flink-related dependencies 'provided' to not have the transitive dependencies promoted Drops the unnecessary dependency on the Hadoop artifact Adds directly referenced libraries, like jackson Deactivates default logging of tests commit a14137c109e73738a3d1f89a3d99e2fd2a799219 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:32:03Z [build] Reduce flink-avro's compile dependency from 'flink-java' to 'flink-core' commit 32e8574498d7963e2ab58f1530b41a6853f23601 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:35:43Z FLINK-6514 [build] Remove 'flink-shaded-hadoop2' from 'flink-dist' via exclusions This is more tedious/manual than setting it to 'provided' once, but it is also correct. For example, in the case of Hadoop 2.3, having 'flink-shaded-hadoop2' as 'provided' removes other needed dependencies as well, such as 'org.codehaus.jackson' from avro. commit efaed902a78c6a7f236e0dad4f72ed7ae8bad1c0 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T11:52:25Z FLINK-6514 [build] Merge bin and lib assembly commit df8efd5cdadaeef9323472f20871871c94d14af5 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:00:03Z FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user StephanEwen opened a pull request:

          https://github.com/apache/flink/pull/3877

          [backport] FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly

          Backport of #3876 to `release-1.3`

          This fixes the issue that Flink cannot be started locally if built with Maven 3.3+

          There are two big fixes in this pull request, because they do not build/pass tests individually. The wrong Mesos dependencies where the reason that the broken Hadoop fat jar building actually passed the Yarn tests.

          1. Hadoop Uber Jar
          • This builds a proper Hadoop Uber Jar with all of Hadoop's needed dependencies. The prior build was missing many important dependencies in the Hadoop Uber Jar.
          • The Hadoop-jar is no longer excluded in `flink-dist` via setting the dependency to `provided`, but by explicit exclusion. That way, Hadoop's transitive dependencies are not excluded from other dependencies as well. Before this patch, various decompression and Avro were broken in a Flink build, due to accidental exclusion of their transitive dependencies.
          1. Dependency fixing
          • This also fixes the dependencies of `flink-mesos`, which made all of Hadoop's transitive dependencies its own dependencies, by promoting them during shading. That way, Flink had various unnecessary dependencies in its `flink-dist` jar.
          • Incidentally, that brought Hadoop's accidentally excluded dependencies back in, but into the `flink-dist` jar, not the `shaded-hadoop2` jar.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/StephanEwen/incubator-flink fix_fat_jar_13

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3877.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3877


          commit ac15bc32a3d786b50e8864a903d31d0b3e0c3042
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T13:12:10Z

          [hotfix] [build] Drop transitive jersey/jettison/servlet dependencies pulled via Hadoop

          commit 5e94787ce4e14f5da88e71418200b4bbe517483b
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T13:12:50Z

          FLINK-6546 [build] Fix dependencies of flink-mesos

          • This makes all flink-related dependencies 'provided' to not have the
            transitive dependencies promoted
          • Drops the unnecessary dependency on the Hadoop artifact
          • Adds directly referenced libraries, like jackson
          • Deactivates default logging of tests

          commit b568ccfdf7366056d29ee43d14c606cfc4448bab
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:32:03Z

          [build] Reduce flink-avro's compile dependency from 'flink-java' to 'flink-core'

          commit 84c150a5798f029bb9aced998ad6b81dd8dc8de5
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:35:43Z

          FLINK-6514 [build] Remove 'flink-shaded-hadoop2' from 'flink-dist' via exclusions

          This is more tedious/manual than setting it to 'provided' once, but it
          is also correct.

          For example, in the case of Hadoop 2.3, having 'flink-shaded-hadoop2' as 'provided'
          removes other needed dependencies as well, such as 'org.codehaus.jackson' from avro.

          commit 99658870865c15ad0996066cf94c721f30bc86ca
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T11:52:25Z

          FLINK-6514 [build] Merge bin and lib assembly

          commit 93e37c666aba50988a48b9273d7b531434c5d5b1
          Author: Stephan Ewen <sewen@apache.org>
          Date: 2017-05-11T15:00:03Z

          FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user StephanEwen opened a pull request: https://github.com/apache/flink/pull/3877 [backport] FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly Backport of #3876 to `release-1.3` This fixes the issue that Flink cannot be started locally if built with Maven 3.3+ There are two big fixes in this pull request, because they do not build/pass tests individually. The wrong Mesos dependencies where the reason that the broken Hadoop fat jar building actually passed the Yarn tests. Hadoop Uber Jar This builds a proper Hadoop Uber Jar with all of Hadoop's needed dependencies. The prior build was missing many important dependencies in the Hadoop Uber Jar. The Hadoop-jar is no longer excluded in `flink-dist` via setting the dependency to `provided`, but by explicit exclusion. That way, Hadoop's transitive dependencies are not excluded from other dependencies as well. Before this patch, various decompression and Avro were broken in a Flink build, due to accidental exclusion of their transitive dependencies. Dependency fixing This also fixes the dependencies of `flink-mesos`, which made all of Hadoop's transitive dependencies its own dependencies, by promoting them during shading. That way, Flink had various unnecessary dependencies in its `flink-dist` jar. Incidentally, that brought Hadoop's accidentally excluded dependencies back in, but into the `flink-dist` jar, not the `shaded-hadoop2` jar. You can merge this pull request into a Git repository by running: $ git pull https://github.com/StephanEwen/incubator-flink fix_fat_jar_13 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3877.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3877 commit ac15bc32a3d786b50e8864a903d31d0b3e0c3042 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T13:12:10Z [hotfix] [build] Drop transitive jersey/jettison/servlet dependencies pulled via Hadoop commit 5e94787ce4e14f5da88e71418200b4bbe517483b Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T13:12:50Z FLINK-6546 [build] Fix dependencies of flink-mesos This makes all flink-related dependencies 'provided' to not have the transitive dependencies promoted Drops the unnecessary dependency on the Hadoop artifact Adds directly referenced libraries, like jackson Deactivates default logging of tests commit b568ccfdf7366056d29ee43d14c606cfc4448bab Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:32:03Z [build] Reduce flink-avro's compile dependency from 'flink-java' to 'flink-core' commit 84c150a5798f029bb9aced998ad6b81dd8dc8de5 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:35:43Z FLINK-6514 [build] Remove 'flink-shaded-hadoop2' from 'flink-dist' via exclusions This is more tedious/manual than setting it to 'provided' once, but it is also correct. For example, in the case of Hadoop 2.3, having 'flink-shaded-hadoop2' as 'provided' removes other needed dependencies as well, such as 'org.codehaus.jackson' from avro. commit 99658870865c15ad0996066cf94c721f30bc86ca Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T11:52:25Z FLINK-6514 [build] Merge bin and lib assembly commit 93e37c666aba50988a48b9273d7b531434c5d5b1 Author: Stephan Ewen <sewen@apache.org> Date: 2017-05-11T15:00:03Z FLINK-6514 [build] Create a proper separate Hadoop uber jar for 'flink-dist' assembly
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3876#discussion_r116157373

          — Diff: flink-dist/pom.xml —
          @@ -403,15 +403,6 @@ under the License.
          </configuration>
          </plugin>

          • <!-- binary compatibility checks -->
              • End diff –

          This is removed because we have it in all the modules itself?

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/3876#discussion_r116157373 — Diff: flink-dist/pom.xml — @@ -403,15 +403,6 @@ under the License. </configuration> </plugin> <!-- binary compatibility checks --> End diff – This is removed because we have it in all the modules itself?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3877#discussion_r116157591

          — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml —
          @@ -0,0 +1,129 @@
          +<?xml version="1.0" encoding="UTF-8"?>
          +<!--
          +Licensed to the Apache Software Foundation (ASF) under one
          +or more contributor license agreements. See the NOTICE file
          +distributed with this work for additional information
          +regarding copyright ownership. The ASF licenses this file
          +to you under the Apache License, Version 2.0 (the
          +"License"); you may not use this file except in compliance
          +with the License. You may obtain a copy of the License at
          +
          + http://www.apache.org/licenses/LICENSE-2.0
          +
          +Unless required by applicable law or agreed to in writing,
          +software distributed under the License is distributed on an
          +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
          +KIND, either express or implied. See the License for the
          +specific language governing permissions and limitations
          +under the License.
          +-->
          +
          +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
          +
          + <modelVersion>4.0.0</modelVersion>
          +
          + <parent>
          + <groupId>org.apache.flink</groupId>
          + <artifactId>flink-shaded-hadoop</artifactId>
          + <version>1.4-SNAPSHOT</version>
          + <relativePath>..</relativePath>
          + </parent>
          +
          + <artifactId>flink-shaded-hadoop2-uber</artifactId>
          + <name>flink-shaded-hadoop2-uber</name>
          +
          + <packaging>jar</packaging>
          +
          + <!--
          + the only dependency if the 'flink-shaded-hadoop2' artifact, out
          — End diff –

          Typo: "dependency is the ..."

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/3877#discussion_r116157591 — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml — @@ -0,0 +1,129 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> + + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-shaded-hadoop</artifactId> + <version>1.4-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <artifactId>flink-shaded-hadoop2-uber</artifactId> + <name>flink-shaded-hadoop2-uber</name> + + <packaging>jar</packaging> + + <!-- + the only dependency if the 'flink-shaded-hadoop2' artifact, out — End diff – Typo: "dependency is the ..."
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3876#discussion_r116157739

          — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml —
          @@ -0,0 +1,129 @@
          +<?xml version="1.0" encoding="UTF-8"?>
          +<!--
          +Licensed to the Apache Software Foundation (ASF) under one
          +or more contributor license agreements. See the NOTICE file
          +distributed with this work for additional information
          +regarding copyright ownership. The ASF licenses this file
          +to you under the Apache License, Version 2.0 (the
          +"License"); you may not use this file except in compliance
          +with the License. You may obtain a copy of the License at
          +
          + http://www.apache.org/licenses/LICENSE-2.0
          +
          +Unless required by applicable law or agreed to in writing,
          +software distributed under the License is distributed on an
          +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
          +KIND, either express or implied. See the License for the
          +specific language governing permissions and limitations
          +under the License.
          +-->
          +
          +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
          +
          + <modelVersion>4.0.0</modelVersion>
          +
          + <parent>
          + <groupId>org.apache.flink</groupId>
          + <artifactId>flink-shaded-hadoop</artifactId>
          + <version>1.4-SNAPSHOT</version>
          + <relativePath>..</relativePath>
          + </parent>
          +
          + <artifactId>flink-shaded-hadoop2-uber</artifactId>
          + <name>flink-shaded-hadoop2-uber</name>
          +
          + <packaging>jar</packaging>
          +
          + <!--
          + the only dependency if the 'flink-shaded-hadoop2' artifact, out
          — End diff –

          Typo "only dependency if ..."

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on a diff in the pull request: https://github.com/apache/flink/pull/3876#discussion_r116157739 — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml — @@ -0,0 +1,129 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> + + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-shaded-hadoop</artifactId> + <version>1.4-SNAPSHOT</version> + <relativePath>..</relativePath> + </parent> + + <artifactId>flink-shaded-hadoop2-uber</artifactId> + <name>flink-shaded-hadoop2-uber</name> + + <packaging>jar</packaging> + + <!-- + the only dependency if the 'flink-shaded-hadoop2' artifact, out — End diff – Typo "only dependency if ..."
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on the issue:

          https://github.com/apache/flink/pull/3877

          The only diff from #3876 is the dropping of the jersey/jettison/servlet deps, right?

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/3877 The only diff from #3876 is the dropping of the jersey/jettison/servlet deps, right?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3876#discussion_r116190819

          — Diff: flink-dist/pom.xml —
          @@ -403,15 +403,6 @@ under the License.
          </configuration>
          </plugin>

          • <!-- binary compatibility checks -->
              • End diff –

          I think this is an artifact from the time when the checker was defined in the root pom and we disabled it for all modules that don't need checking.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/3876#discussion_r116190819 — Diff: flink-dist/pom.xml — @@ -403,15 +403,6 @@ under the License. </configuration> </plugin> <!-- binary compatibility checks --> End diff – I think this is an artifact from the time when the checker was defined in the root pom and we disabled it for all modules that don't need checking.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3876

          The change looks good for me.
          I'll verify it with my Cloudera VM to make sure it works with provided hadoop versions as well.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3876 The change looks good for me. I'll verify it with my Cloudera VM to make sure it works with provided hadoop versions as well.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3876

          This PR works in my VM. There's just some weird behavior when starting Flink on YARN.
          I'll cross check with AWS EMR to see if its a YARN or a VM/Cloudera issue.

          I wanted to check on EMR anyways for the release

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3876 This PR works in my VM. There's just some weird behavior when starting Flink on YARN. I'll cross check with AWS EMR to see if its a YARN or a VM/Cloudera issue. I wanted to check on EMR anyways for the release
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on a diff in the pull request:

          https://github.com/apache/flink/pull/3877#discussion_r116227807

          — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml —
          @@ -0,0 +1,129 @@
          +<?xml version="1.0" encoding="UTF-8"?>
          +<!--
          +Licensed to the Apache Software Foundation (ASF) under one
          +or more contributor license agreements. See the NOTICE file
          +distributed with this work for additional information
          +regarding copyright ownership. The ASF licenses this file
          +to you under the Apache License, Version 2.0 (the
          +"License"); you may not use this file except in compliance
          +with the License. You may obtain a copy of the License at
          +
          + http://www.apache.org/licenses/LICENSE-2.0
          +
          +Unless required by applicable law or agreed to in writing,
          +software distributed under the License is distributed on an
          +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
          +KIND, either express or implied. See the License for the
          +specific language governing permissions and limitations
          +under the License.
          +-->
          +
          +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
          +
          + <modelVersion>4.0.0</modelVersion>
          +
          + <parent>
          + <groupId>org.apache.flink</groupId>
          + <artifactId>flink-shaded-hadoop</artifactId>
          + <version>1.4-SNAPSHOT</version>
          — End diff –

          This needs to be 1.3-SNAPSHOT.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/3877#discussion_r116227807 — Diff: flink-shaded-hadoop/flink-shaded-hadoop2-uber/pom.xml — @@ -0,0 +1,129 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> + + <modelVersion>4.0.0</modelVersion> + + <parent> + <groupId>org.apache.flink</groupId> + <artifactId>flink-shaded-hadoop</artifactId> + <version>1.4-SNAPSHOT</version> — End diff – This needs to be 1.3-SNAPSHOT.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3876

          +1 to merge

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3876 +1 to merge
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3876

          I'm merging this change now

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3876 I'm merging this change now
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3876

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3876
          Show
          rmetzger Robert Metzger added a comment - Merged for 1.4 in http://git-wip-us.apache.org/repos/asf/flink/commit/aa135914 http://git-wip-us.apache.org/repos/asf/flink/commit/3c77812e http://git-wip-us.apache.org/repos/asf/flink/commit/63bbb4f5 http://git-wip-us.apache.org/repos/asf/flink/commit/e4cdf125 http://git-wip-us.apache.org/repos/asf/flink/commit/ae423e1d
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3877

          Merging this change ...

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3877 Merging this change ...
          Show
          rmetzger Robert Metzger added a comment - Merged for 1.3: http://git-wip-us.apache.org/repos/asf/flink/commit/e8d9f238 http://git-wip-us.apache.org/repos/asf/flink/commit/b32b5867 http://git-wip-us.apache.org/repos/asf/flink/commit/38ff7b95 http://git-wip-us.apache.org/repos/asf/flink/commit/24d4688c http://git-wip-us.apache.org/repos/asf/flink/commit/5d05c18d
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/3877

          @StephanEwen can you manually close the PR?
          Thx.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/3877 @StephanEwen can you manually close the PR? Thx.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StephanEwen closed the pull request at:

          https://github.com/apache/flink/pull/3877

          Show
          githubbot ASF GitHub Bot added a comment - Github user StephanEwen closed the pull request at: https://github.com/apache/flink/pull/3877

            People

            • Assignee:
              StephanEwen Stephan Ewen
              Reporter:
              aljoscha Aljoscha Krettek
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development