Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.0
    • Component/s: build
    • Labels:

      Description

      SAMZA-430 updated Samza to remove legacy Scala 2.9.2 code. We should upgrade Samza to build and default to Scala 2.11. This will require us waiting for Kafka 0.8.2 to be released, as it appears to be the first version to support Scala 2.11 (KAFKA-1419).

        Issue Links

          Activity

          Hide
          kanterov Gleb Kanterov added a comment -

          Hi,

          I'm running master branch with Scala 2.11 and Kafka 0.8.2-beta and haven't found issues with it for running my jobs in dev environment, I've attached patch for it.

          Thanks,
          Gleb

          Show
          kanterov Gleb Kanterov added a comment - Hi, I'm running master branch with Scala 2.11 and Kafka 0.8.2-beta and haven't found issues with it for running my jobs in dev environment, I've attached patch for it. Thanks, Gleb
          Hide
          criccomini Chris Riccomini added a comment -

          Gleb Kanterov, this looks good. Could you please update bin/check-all.sh as well?

          Also, note: committing this will result in us having to deprecate JDK 6, since Scalatra 2.3 is built against JDK 7.

          Show
          criccomini Chris Riccomini added a comment - Gleb Kanterov , this looks good. Could you please update bin/check-all.sh as well? Also, note: committing this will result in us having to deprecate JDK 6, since Scalatra 2.3 is built against JDK 7.
          Hide
          criccomini Chris Riccomini added a comment -

          Ran tests against current patch, and all passed using JDK 7. Failed on JDK 6 on Scalatra test.

          Show
          criccomini Chris Riccomini added a comment - Ran tests against current patch, and all passed using JDK 7. Failed on JDK 6 on Scalatra test.
          Hide
          kanterov Gleb Kanterov added a comment -

          Chris Riccomini, I've updated the patch. I've updated bin/check-all.sh, removed warnings caused by deprecated API, updated docs and added Travis CI build for 2.11.

          If it worths, Probably we could use raw servlet API to exclude scalatra dependency and have JDK 6 support. Personally I don't have any concern on deprecating it.

          Show
          kanterov Gleb Kanterov added a comment - Chris Riccomini , I've updated the patch. I've updated bin/check-all.sh, removed warnings caused by deprecated API, updated docs and added Travis CI build for 2.11. If it worths, Probably we could use raw servlet API to exclude scalatra dependency and have JDK 6 support. Personally I don't have any concern on deprecating it.
          Show
          criccomini Chris Riccomini added a comment - Opened a discussion on the mailing list about this issue: http://mail-archives.apache.org/mod_mbox/incubator-samza-dev/201412.mbox/%3CD0AC655A.38469%25criccomini%40linkedin.com%3E
          Hide
          bandrews bennett andrews added a comment -

          Chris Riccomini Any update on this? Would be nice for 0.9.0 so we no longer have to maintain our own builds.

          Show
          bandrews bennett andrews added a comment - Chris Riccomini Any update on this? Would be nice for 0.9.0 so we no longer have to maintain our own builds.
          Hide
          criccomini Chris Riccomini added a comment - - edited

          bennett andrews, sorry for the inconvenience.

          We're blocked on SAMZA-455. Due to some transitive dependencies, we can't upgrade to 2.11 until we abandon JDK6 support. We will be ditching JDK6 in June of this year. Likely 0.10.0 will be the last JDK6 release.

          The transitive dependencies that I'm talking about are Scalatra, which requires us to upgrade to 2.3.0. This version, AFAIK, is built against JDK 7.

          Show
          criccomini Chris Riccomini added a comment - - edited bennett andrews , sorry for the inconvenience. We're blocked on SAMZA-455 . Due to some transitive dependencies, we can't upgrade to 2.11 until we abandon JDK6 support. We will be ditching JDK6 in June of this year. Likely 0.10.0 will be the last JDK6 release. The transitive dependencies that I'm talking about are Scalatra, which requires us to upgrade to 2.3.0. This version, AFAIK, is built against JDK 7.
          Hide
          closeuris Yan Fang added a comment -

          per bennett andrews's comment in SAMZA-455, does anything block us to drop the java 6 support in 0.9.0 release? ( I know, the announcement is in the website.) Seems no current users in our community are using jdk 6 ? from the mailing list,
          https://mail-archives.apache.org/mod_mbox/incubator-samza-dev/201502.mbox/%3C962D3CAB94174A4E9B771B88A9DFE7B10F1E1BF0@SJEXMB02.Tivo.com%3E

          Show
          closeuris Yan Fang added a comment - per bennett andrews 's comment in SAMZA-455 , does anything block us to drop the java 6 support in 0.9.0 release? ( I know, the announcement is in the website.) Seems no current users in our community are using jdk 6 ? from the mailing list, https://mail-archives.apache.org/mod_mbox/incubator-samza-dev/201502.mbox/%3C962D3CAB94174A4E9B771B88A9DFE7B10F1E1BF0@SJEXMB02.Tivo.com%3E
          Hide
          criccomini Chris Riccomini added a comment -

          Yan Fang, AFAIK, it's just a policy thing. We discussed at length, and there were several dissenting views (Jay Kreps, E. Sammer, Martin Kleppmann), who preferred to at minimum, set a date, and communicate it broadly. We've done that (June, website update, meetup notice). At this point, it's just two months out, but the broad notifications just started a month ago. IMO, it's probably worth just waiting the two months to be good citizens. After 0.9.0 release, I think we should just move master over immediately, and the next (0.10.0) release will fall right near June, so it can be JDK7.

          Show
          criccomini Chris Riccomini added a comment - Yan Fang , AFAIK, it's just a policy thing. We discussed at length, and there were several dissenting views ( Jay Kreps , E. Sammer , Martin Kleppmann ), who preferred to at minimum, set a date, and communicate it broadly. We've done that (June, website update, meetup notice). At this point, it's just two months out, but the broad notifications just started a month ago. IMO, it's probably worth just waiting the two months to be good citizens. After 0.9.0 release, I think we should just move master over immediately, and the next (0.10.0) release will fall right near June, so it can be JDK7.
          Hide
          closeuris Yan Fang added a comment -

          yes, that sound pretty good to me. Thank you.

          Show
          closeuris Yan Fang added a comment - yes, that sound pretty good to me. Thank you.
          Hide
          timcharper Tim Harper added a comment - - edited

          I came looking for this. Is there a branch somewhere with the Scala 2.11 patch(es) applied, so I can at least publish local?

          Show
          timcharper Tim Harper added a comment - - edited I came looking for this. Is there a branch somewhere with the Scala 2.11 patch(es) applied, so I can at least publish local?
          Hide
          timcharper Tim Harper added a comment -

          I've rebased Gleb's patch against the latest master, bf094c8e5c4038b158811f51e8ca523bb7c1b182. Also, I updated the Kafka dependency to 0.8.2.1.

          Using this patch, I was able to compile against Scala 2.11; all of the tests passed.

          Show
          timcharper Tim Harper added a comment - I've rebased Gleb's patch against the latest master, bf094c8e5c4038b158811f51e8ca523bb7c1b182 . Also, I updated the Kafka dependency to 0.8.2.1 . Using this patch, I was able to compile against Scala 2.11; all of the tests passed.
          Hide
          closeuris Yan Fang added a comment -

          Anything we need to consider to upgrade to 2.11?

          Show
          closeuris Yan Fang added a comment - Anything we need to consider to upgrade to 2.11?
          Hide
          navina Navina Ramesh added a comment -

          Afaik, Yi Pan (Data Infrastructure) tried to upgrade to scala 2.11 and ran into issues.
          There wasn't enough motivations to get it working, esp. because everyone seemed to agree to move away from scala.

          Show
          navina Navina Ramesh added a comment - Afaik, Yi Pan (Data Infrastructure) tried to upgrade to scala 2.11 and ran into issues. There wasn't enough motivations to get it working, esp. because everyone seemed to agree to move away from scala.
          Hide
          craigtmc Craig McIntosh added a comment -

          Has a decision to move away from Scala been made and is there a timeframe for the move? If not a 2.11 or 2.12 version would be really helpful.

          My concern is that with 2.12 coming up soon, other Scala libraries we use in our Samza jobs will stop cross compiling to 2.10 in the near future.

          Show
          craigtmc Craig McIntosh added a comment - Has a decision to move away from Scala been made and is there a timeframe for the move? If not a 2.11 or 2.12 version would be really helpful. My concern is that with 2.12 coming up soon, other Scala libraries we use in our Samza jobs will stop cross compiling to 2.10 in the near future.
          Hide
          slevinbe Yennick Trevels added a comment -

          I've got the same concern as Craig. For example, the scala-logging library has already stopped cross compiling to 2.10.
          Any update on this issue? Seems like Tim Harper was able to compile the codebase last year against 2.11.

          Show
          slevinbe Yennick Trevels added a comment - I've got the same concern as Craig. For example, the scala-logging library has already stopped cross compiling to 2.10. Any update on this issue? Seems like Tim Harper was able to compile the codebase last year against 2.11.
          Hide
          Geoff Bransford-Koons Geoffrey R Bransford-Koons added a comment -

          We're in the same boat. Some of our Samza tasks have dependencies that need Scala 2.11.

          Any update would be welcome, specially if the decision is made not to support Scala anymore.

          Show
          Geoff Bransford-Koons Geoffrey R Bransford-Koons added a comment - We're in the same boat. Some of our Samza tasks have dependencies that need Scala 2.11. Any update would be welcome, specially if the decision is made not to support Scala anymore.
          Hide
          craigtmc Craig McIntosh added a comment -

          I have a branch based of 0.11.0 that works with 2.11. Our internal tests and samza tests pass.

          https://github.com/craigtmc/samza/tree/scala-2.11

          Show
          craigtmc Craig McIntosh added a comment - I have a branch based of 0.11.0 that works with 2.11. Our internal tests and samza tests pass. https://github.com/craigtmc/samza/tree/scala-2.11
          Hide
          nickpan47 Yi Pan (Data Infrastructure) added a comment -

          Thanks, Craig McIntosh, could you open a pull request?

          Show
          nickpan47 Yi Pan (Data Infrastructure) added a comment - Thanks, Craig McIntosh , could you open a pull request?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user craigtmc opened a pull request:

          https://github.com/apache/samza/pull/28

          SAMZA-469: Update Scala version to 2.11

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/craigtmc/samza master-scala-2.11

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/samza/pull/28.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #28


          commit 94f1f7cab999e1ae2dc63239c9e8e0ef4227f2f8
          Author: McIntosh, Craig <craig.mcintosh@ncr.com>
          Date: 2016-12-01T18:48:30Z

          SAMZA-469: Update Scala version to 2.11


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user craigtmc opened a pull request: https://github.com/apache/samza/pull/28 SAMZA-469 : Update Scala version to 2.11 You can merge this pull request into a Git repository by running: $ git pull https://github.com/craigtmc/samza master-scala-2.11 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/samza/pull/28.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #28 commit 94f1f7cab999e1ae2dc63239c9e8e0ef4227f2f8 Author: McIntosh, Craig <craig.mcintosh@ncr.com> Date: 2016-12-01T18:48:30Z SAMZA-469 : Update Scala version to 2.11
          Hide
          craigtmc Craig McIntosh added a comment -
          Show
          craigtmc Craig McIntosh added a comment - PR submitted. https://github.com/apache/samza/pull/28
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/samza/pull/28

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/28
          Hide
          nickpan47 Yi Pan (Data Infrastructure) added a comment -

          PR 28 merged and submitted. Thanks!

          Show
          nickpan47 Yi Pan (Data Infrastructure) added a comment - PR 28 merged and submitted. Thanks!

            People

            • Assignee:
              craigtmc Craig McIntosh
              Reporter:
              criccomini Chris Riccomini
            • Votes:
              8 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development