Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-2000

Add Spark 2.2.0 as supported binary release

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Spark 2.2.0 was recently released. This JIRA is for making sure Mahout is supported with the spark 2.2.0 backend, and also publishing the binary.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rawkintrevo commented on the issue:

          https://github.com/apache/mahout/pull/332

          @dustinvanstee looked good. The 'gotchya' was Spark 2.2.0 officially dropped java 7 support, so those binaries are compiled with java8[1]. I found that out by googling the error in travis[2]

          Because I'm curious- try changing [3] to `openjdk8` and see what happens. Will need further discusion about releasing Java8 for spark2 compatability on dev, but first lets see if it works.

          [1] https://people.apache.org/~pwendell/spark-releases/latest/
          [2] https://stackoverflow.com/questions/18161907/exception-in-thread-main-java-lang-unsupportedclassversionerror-unsupported-m
          [3] https://github.com/apache/mahout/pull/332/files#diff-354f30a63fb0907d4ad57269548329e3R59

          Show
          githubbot ASF GitHub Bot added a comment - Github user rawkintrevo commented on the issue: https://github.com/apache/mahout/pull/332 @dustinvanstee looked good. The 'gotchya' was Spark 2.2.0 officially dropped java 7 support, so those binaries are compiled with java8 [1] . I found that out by googling the error in travis [2] Because I'm curious- try changing [3] to `openjdk8` and see what happens. Will need further discusion about releasing Java8 for spark2 compatability on dev, but first lets see if it works. [1] https://people.apache.org/~pwendell/spark-releases/latest/ [2] https://stackoverflow.com/questions/18161907/exception-in-thread-main-java-lang-unsupportedclassversionerror-unsupported-m [3] https://github.com/apache/mahout/pull/332/files#diff-354f30a63fb0907d4ad57269548329e3R59
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rawkintrevo commented on the issue:

          https://github.com/apache/mahout/pull/332

          I checked that this does fail when trying to build with `-Pspark2.2` in Java7. The error is non descript though, and it fails in Mahout Math.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rawkintrevo commented on the issue: https://github.com/apache/mahout/pull/332 I checked that this does fail when trying to build with `-Pspark2.2` in Java7. The error is non descript though, and it fails in Mahout Math.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rawkintrevo commented on the issue:

          https://github.com/apache/mahout/pull/332

          It's looking good, could you squash your commits and give me a chance to look it over once more pls.

          Show
          githubbot ASF GitHub Bot added a comment - Github user rawkintrevo commented on the issue: https://github.com/apache/mahout/pull/332 It's looking good, could you squash your commits and give me a chance to look it over once more pls.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dustinvanstee commented on the issue:

          https://github.com/apache/mahout/pull/332

          Closing this pull request and opening a new one.

          Show
          githubbot ASF GitHub Bot added a comment - Github user dustinvanstee commented on the issue: https://github.com/apache/mahout/pull/332 Closing this pull request and opening a new one.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dustinvanstee closed the pull request at:

          https://github.com/apache/mahout/pull/332

          Show
          githubbot ASF GitHub Bot added a comment - Github user dustinvanstee closed the pull request at: https://github.com/apache/mahout/pull/332
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user dustinvanstee opened a pull request:

          https://github.com/apache/mahout/pull/335

          MAHOUT-2000 [WIP] Add maven profile for Spark 2.2

              1. Purpose of PR: Add spark 2.2 into travisCI framework and add a maven spark 2.2 profile.
              1. Important ToDos
                Please mark each with an "x"
          • [x] A JIRA ticket exists (if not, please create this first)https://issues.apache.org/jira/browse/ZEPPELIN/
          • [ x] Title of PR is "MAHOUT-XXXX Brief Description of Changes" where XXXX is the JIRA number.
          • [ ] Created unit tests where appropriate
          • [ ] Added licenses correct on newly added files
          • [x ] Assigned JIRA to self
          • [ ] Added documentation in scala docs/java docs, and to website
          • [ ] Successfully built and ran all unit tests, verified that all tests pass locally.

          If all of these things aren't complete, but you still feel it is
          appropriate to open a PR, please add [WIP] after MAHOUT-XXXX before the
          descriptions- e.g. "MAHOUT-XXXX [WIP] Description of Change"

          Does this change break earlier versions?

          Is this the beginning of a larger project for which a feature branch should be made?

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/dustinvanstee/mahout MAHOUT-2000

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/mahout/pull/335.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #335


          commit e2bbcfce13d960c3eb1240d2607c95e4a32e5144
          Author: vanstee <vanstee@apache.org>
          Date: 2017-07-20T15:48:35Z

          spark 2.2 support added to pom.xml and travis.yml


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user dustinvanstee opened a pull request: https://github.com/apache/mahout/pull/335 MAHOUT-2000 [WIP] Add maven profile for Spark 2.2 Purpose of PR: Add spark 2.2 into travisCI framework and add a maven spark 2.2 profile. Important ToDos Please mark each with an "x" [x] A JIRA ticket exists (if not, please create this first) https://issues.apache.org/jira/browse/ZEPPELIN/ [ x] Title of PR is "MAHOUT-XXXX Brief Description of Changes" where XXXX is the JIRA number. [ ] Created unit tests where appropriate [ ] Added licenses correct on newly added files [x ] Assigned JIRA to self [ ] Added documentation in scala docs/java docs, and to website [ ] Successfully built and ran all unit tests, verified that all tests pass locally. If all of these things aren't complete, but you still feel it is appropriate to open a PR, please add [WIP] after MAHOUT-XXXX before the descriptions- e.g. "MAHOUT-XXXX [WIP] Description of Change" Does this change break earlier versions? Is this the beginning of a larger project for which a feature branch should be made? You can merge this pull request into a Git repository by running: $ git pull https://github.com/dustinvanstee/mahout MAHOUT-2000 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/mahout/pull/335.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #335 commit e2bbcfce13d960c3eb1240d2607c95e4a32e5144 Author: vanstee <vanstee@apache.org> Date: 2017-07-20T15:48:35Z spark 2.2 support added to pom.xml and travis.yml
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user rawkintrevo commented on the issue:

          https://github.com/apache/mahout/pull/335

          This is looking good- the only other thing I'd add is there should be a building Mahout webpage that callls out that the Spark 2.2 profile will build with Java8 / Fail on Java7 (and Hadoop 2.6, instead of 2.4)

          Show
          githubbot ASF GitHub Bot added a comment - Github user rawkintrevo commented on the issue: https://github.com/apache/mahout/pull/335 This is looking good- the only other thing I'd add is there should be a building Mahout webpage that callls out that the Spark 2.2 profile will build with Java8 / Fail on Java7 (and Hadoop 2.6, instead of 2.4)

            People

            • Assignee:
              rawkintrevo Trevor Grant
              Reporter:
              vanstee Dustin VanStee
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:

                Development