Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4816

Maven profile netlib-lgpl does not work

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.1, 1.4.2
    • Component/s: Build
    • Labels:
      None
    • Environment:

      maven 3.0.5 / Ubuntu

      Description

      When doing what the documentation recommends to recompile Spark with Netlib Native system binding (i.e. to bind with openblas or, in my case, MKL),

      mvn -Pnetlib-lgpl -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package

      The resulting assembly jar still lacked the netlib-system class. (I checked the content of spark-assembly...jar)

      When forcing the netlib-lgpl profile in MLLib package to be active, the jar is correctly built.

      So I guess it's a problem with the way maven passes profiles activitations to children modules.

      Also, despite the documentation claiming that if the job's jar contains netlib with necessary bindings, it should works, it does not. The classloader must be unhappy with two occurrences of netlib ?

        Attachments

          Activity

            People

            • Assignee:
              srowen Sean Owen
              Reporter:
              guillaumepitel Guillaume Pitel
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: