Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-2086

use consistent SBT resolvable jar naming scheme with the correct convention

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 14.1
    • Fix Version/s: 14.1
    • Component/s: None
    • Labels:
      None

      Description

       currently we build jars for each module, module_x, in its respective target directory.. naming that module as

       $MAHOUT_HOME/module_x/target/mahout-module-x_${scala.compat.version}-{project.version}.jar.

      and then copy using an ant plugin via the pom.xml each jar file to the /lib directory.  Pat Ferrel  has had issues resolving via SBT.  

      It seems that our artifacts should be resolvable by SBT e.g, for core math, Scala_2.12, mahout v14.1:

       libraryDependencies += "org.apache.mahout" % "mahout-core_2.12" % "14.1"

      Currently we have our Spark build pegged to 2.4.3.

      Had trouble finding A naming convention for artifacts in the Scala docs.  However, this Spark style guide[1], makes the case for:

      module-x_scalaVersion-projectVersion.jar

       

      This means that only the distribution artifacts jars need to be changed, If anything.

       

       

      JAR Files

      You can build projects that support multiple Spark versions or just a single Spark version.

      Projects that support a single Spark version

      JAR files built for a specific Spark version should be named like this:

       

      spark-testing-base_2.11-2.1.0_0.6.0.jar

      Generically:

       

      spark-testing-base_scalaVersion-sparkVersion_projectVersion.jar

      {{}}

      If you're using sbt assembly, you can use the following line of code to build a JAR file using the correct naming conventions.
      assemblyJarName in assembly := s"${name.value}${scalaBinaryVersion.value}-${sparkVersion.value}${version.value}.jar"
      If you're using sbt package, you can add this code to your build.sbt file to generate a JAR file that follows the naming conventions.
      artifactName :=

      { (sv: ScalaVersion, module: ModuleID, artifact: Artifact) => artifact.name + "_" + sv.binary + "-" + sparkVersion.value + "_" + module.revision + "." + artifact.extension }

      Projects that support multiple Spark versions

      JAR files built for multiple Spark version should be named like this:

      spark-testing-base_2.11-0.6.0.jar

      Generically:

      spark-testing-base_scalaVersion-projectVersion.jar

      {{}}

      If you're using sbt assembly, you can use the following line of code to build a JAR file using the correct naming conventions.
      assemblyJarName in assembly := s"${name.value}_${scalaBinaryVersion.value}-${version.value}.jar"
      If you're using sbt package, you can add this code to your build.sbt file to generate a JAR file that follows the naming conventions.
      artifactName :=

      { (sv: ScalaVersion, module: ModuleID, artifact: Artifact) => artifact.name + "_" + sv.binary + "-" + module.revision + "." + artifact.extension }

       

       

      [1] https://github.com/mrpowers/spark-style-guide

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Andrew_Palumbo Andrew Palumbo
                Reporter:
                Andrew_Palumbo Andrew Palumbo
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m