Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0, 1.4.0
    • Fix Version/s: 1.2.2, 1.4.0, 1.3.2
    • Component/s: Tests
    • Labels:
      None

      Description

      As shown in this PR https://github.com/apache/flink/pull/4167 switching to the Trusty image on Travis seems to stabilize the build times.

      We should switch for 1.2, 1.3 and 1.4.

        Issue Links

          Activity

          Hide
          Zentol Chesnay Schepler added a comment -

          1.2: 3fcc959dd4d3c65436b44bb22848cf2565461de2
          1.3: 001df19797e5155f0c6dff06eee796a6d83af19e
          1.4: 958d3762de3c84264199c3c02035ff2f674973ce

          Show
          Zentol Chesnay Schepler added a comment - 1.2: 3fcc959dd4d3c65436b44bb22848cf2565461de2 1.3: 001df19797e5155f0c6dff06eee796a6d83af19e 1.4: 958d3762de3c84264199c3c02035ff2f674973ce
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4207

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4207
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4208

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4208
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4208

          Will merge once travis is done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4208 Will merge once travis is done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4207

          Will merge once travis is done.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4207 Will merge once travis is done.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4182

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4182
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4182

          Will address issues while merging.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4182 Will address issues while merging.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/4208

          +1 to merge once travis has passed

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/4208 +1 to merge once travis has passed
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/4207

          +1 to merge once travis has passed

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/4207 +1 to merge once travis has passed
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on the issue:

          https://github.com/apache/flink/pull/4182

          +1 to merge once my concerns are addressed

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/4182 +1 to merge once my concerns are addressed
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on a diff in the pull request:

          https://github.com/apache/flink/pull/4182#discussion_r124588442

          — Diff: pom.xml —
          @@ -1021,6 +1021,7 @@ under the License.
          <!-- Tools: watchdog -->
          <exclude>tools/artifacts/**</exclude>
          <exclude>tools/flink*/**</exclude>
          + <exclude>apache-maven-3.2.5/**</exclude>
          — End diff –

          Maybe add a comment why this exclusion is needed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/4182#discussion_r124588442 — Diff: pom.xml — @@ -1021,6 +1021,7 @@ under the License. <!-- Tools: watchdog --> <exclude>tools/artifacts/**</exclude> <exclude>tools/flink*/**</exclude> + <exclude>apache-maven-3.2.5/**</exclude> — End diff – Maybe add a comment why this exclusion is needed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rmetzger commented on a diff in the pull request:

          https://github.com/apache/flink/pull/4182#discussion_r124588048

          — Diff: .travis.yml —
          @@ -1,7 +1,8 @@

          1. s3 deployment based on http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/
          1. send to container based infrastructure: http://docs.travis-ci.com/user/workers/container-based-infrastructure/
              • End diff –

          This comment is actually invalid now. Its the "SUDO-ENABLED" infra.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/4182#discussion_r124588048 — Diff: .travis.yml — @@ -1,7 +1,8 @@ s3 deployment based on http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/ send to container based infrastructure: http://docs.travis-ci.com/user/workers/container-based-infrastructure/ End diff – This comment is actually invalid now. Its the "SUDO-ENABLED" infra.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zentol opened a pull request:

          https://github.com/apache/flink/pull/4208

          FLINK-7004 Switch to Travis Trusty image

          Back-port of #4182 for 1.2.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zentol/flink 7004_2

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4208.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4208


          commit 29f7c18147da251143c814a843b1416bb46530fe
          Author: zentol <chesnay@apache.org>
          Date: 2017-06-26T09:44:45Z

          FLINK-7004 Switch to Travis Trusty image

          • enable sudo for more memory
          • increase java heap size
          • replace usage of oraclejdk7 since it is no longer supporter
          • manually install maven 3.2.5

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/4208 FLINK-7004 Switch to Travis Trusty image Back-port of #4182 for 1.2. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 7004_2 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4208.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4208 commit 29f7c18147da251143c814a843b1416bb46530fe Author: zentol <chesnay@apache.org> Date: 2017-06-26T09:44:45Z FLINK-7004 Switch to Travis Trusty image enable sudo for more memory increase java heap size replace usage of oraclejdk7 since it is no longer supporter manually install maven 3.2.5
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zentol opened a pull request:

          https://github.com/apache/flink/pull/4207

          FLINK-7004 Switch to Travis Trusty image

          Back-port of #4182 for 1.3.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zentol/flink 7004_3

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4207.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4207


          commit 7d47d5a807aa33c8d1379ce02b3abb0af8d0e82f
          Author: zentol <chesnay@apache.org>
          Date: 2017-06-26T09:44:45Z

          FLINK-7004 Switch to Travis Trusty image

          • enable sudo for more memory
          • increase java heap size
          • replace usage of oraclejdk7 since it is no longer supporter
          • manually install maven 3.2.5

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/4207 FLINK-7004 Switch to Travis Trusty image Back-port of #4182 for 1.3. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 7004_3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4207.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4207 commit 7d47d5a807aa33c8d1379ce02b3abb0af8d0e82f Author: zentol <chesnay@apache.org> Date: 2017-06-26T09:44:45Z FLINK-7004 Switch to Travis Trusty image enable sudo for more memory increase java heap size replace usage of oraclejdk7 since it is no longer supporter manually install maven 3.2.5
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4182

          huh, looks like there's a problem with hadoop2.6.5 + openjdk7. The `WordCountMapreduceITCase` fails for this profile, and also did so in a previous build (see #4167).
          ```
          Running org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase
          zip file closed
          java.lang.IllegalStateException: zip file closed
          at java.util.zip.ZipFile.ensureOpen(ZipFile.java:634)
          at java.util.zip.ZipFile.getInputStream(ZipFile.java:347)
          at java.util.jar.JarFile.getInputStream(JarFile.java:412)
          at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:162)
          at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:226)
          at javax.xml.parsers.SecuritySupport$4.run(SecuritySupport.java:94)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.xml.parsers.SecuritySupport.getResourceAsStream(SecuritySupport.java:87)
          at javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:283)
          at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:255)
          at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:121)
          at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2467)
          at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2444)
          at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2361)
          at org.apache.hadoop.conf.Configuration.get(Configuration.java:968)
          at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:2010)
          at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:449)
          at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:187)
          at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:168)
          at org.apache.flink.hadoopcompatibility.HadoopInputs.readHadoopFile(HadoopInputs.java:104)
          at org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase.internalRun(WordCountMapreduceITCase.java:78)
          at org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase.testProgram(WordCountMapreduceITCase.java:67)
          at org.apache.flink.test.util.JavaProgramTestBase.testJobWithoutObjectReuse(JavaProgramTestBase.java:180)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:606)
          at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
          at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
          at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
          at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
          at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
          at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
          at org.junit.rules.RunRules.evaluate(RunRules.java:20)
          at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
          at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
          at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
          at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
          at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
          at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
          at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
          at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
          at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
          at org.junit.rules.RunRules.evaluate(RunRules.java:20)
          at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
          at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
          at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
          at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
          at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
          at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
          at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
          at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4182 huh, looks like there's a problem with hadoop2.6.5 + openjdk7. The `WordCountMapreduceITCase` fails for this profile, and also did so in a previous build (see #4167). ``` Running org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase zip file closed java.lang.IllegalStateException: zip file closed at java.util.zip.ZipFile.ensureOpen(ZipFile.java:634) at java.util.zip.ZipFile.getInputStream(ZipFile.java:347) at java.util.jar.JarFile.getInputStream(JarFile.java:412) at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:162) at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:226) at javax.xml.parsers.SecuritySupport$4.run(SecuritySupport.java:94) at java.security.AccessController.doPrivileged(Native Method) at javax.xml.parsers.SecuritySupport.getResourceAsStream(SecuritySupport.java:87) at javax.xml.parsers.FactoryFinder.findJarServiceProvider(FactoryFinder.java:283) at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:255) at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:121) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2467) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2444) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2361) at org.apache.hadoop.conf.Configuration.get(Configuration.java:968) at org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:2010) at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:449) at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:187) at org.apache.hadoop.mapreduce.Job.getInstance(Job.java:168) at org.apache.flink.hadoopcompatibility.HadoopInputs.readHadoopFile(HadoopInputs.java:104) at org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase.internalRun(WordCountMapreduceITCase.java:78) at org.apache.flink.test.hadoop.mapreduce.WordCountMapreduceITCase.testProgram(WordCountMapreduceITCase.java:67) at org.apache.flink.test.util.JavaProgramTestBase.testJobWithoutObjectReuse(JavaProgramTestBase.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zentol opened a pull request:

          https://github.com/apache/flink/pull/4182

          FLINK-7004 Switch to Travis Trusty image

          Cleaned up version of #4167. With this PR we switch to the trusty image on Travis as it appears to have more stable build times.

          Other changes include:

          • run in a sudo-enabled environment for more memory
          • increase java heap size
          • replace oraclejdk7 profile since it is no longer supported (see https://github.com/travis-ci/travis-ci/issues/7884)
          • manually install maven 3.2.5 since trusty works with 3.3.9

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zentol/flink 7004

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4182.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4182


          commit e3ed988581a3f0e747d4a80335f86dd1ace07106
          Author: zentol <chesnay@apache.org>
          Date: 2017-06-26T09:44:45Z

          FLINK-7004 Switch to Travis Trusty image

          • enable sudo for more memory
          • increase java heap size
          • replace usage of oraclejdk7 since it is no longer supporter
          • manually install maven 3.2.5

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/4182 FLINK-7004 Switch to Travis Trusty image Cleaned up version of #4167. With this PR we switch to the trusty image on Travis as it appears to have more stable build times. Other changes include: run in a sudo-enabled environment for more memory increase java heap size replace oraclejdk7 profile since it is no longer supported (see https://github.com/travis-ci/travis-ci/issues/7884 ) manually install maven 3.2.5 since trusty works with 3.3.9 You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 7004 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4182.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4182 commit e3ed988581a3f0e747d4a80335f86dd1ace07106 Author: zentol <chesnay@apache.org> Date: 2017-06-26T09:44:45Z FLINK-7004 Switch to Travis Trusty image enable sudo for more memory increase java heap size replace usage of oraclejdk7 since it is no longer supporter manually install maven 3.2.5

            People

            • Assignee:
              Zentol Chesnay Schepler
              Reporter:
              Zentol Chesnay Schepler
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development