Uploaded image for project: 'PredictionIO (Retired)'
  1. PredictionIO (Retired)
  2. PIO-30

Cross build for different versions of scala and spark

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0-incubating
    • None
    • None

    Description

      The present version of Scala is 2.10 and Spark is 1.4, which is quite old. With Spark 2.0.0 come many performance improvements and features, that people will definitely like to add to their templates. I am also aware that past cannot be ignored and simply dumping 1.x might not be an option for other users.

      I propose setting up a crossbuild in sbt to build with scala 2.10 and Spark 1.6 and a separate one for Scala 2.11 and Spark 2.0. Most of the files will be consistent between versions including API. The problematic ones will be divided between additional source directories: src/main/scala-2.10/ and src/main/scala-2.11/. The dockerized tests should also take the two versions into consideration

      Attachments

        Issue Links

          Activity

            People

              chanlee514 Chan
              ziemin Marcin ZiemiƄski
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: