Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1955

Viennacl jars are not being picked up by the shell startup script

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.13.0
    • Fix Version/s: 0.13.0
    • Component/s: Mahout spark shell
    • Labels:
      None

      Description

      When building for ViennaCL for OpenCL or OpenMP:

      $ mvn clean install -Pviennacl -Phadoop2 -DskipTests
      

      or

      $ mvn clean install -Pviennacl-omp -Phadoop2 -DskipTests
      

      The mahout-native-*_2.10.jar s are not being picked up by the shell which are now in the top-level directory after the build. Fix is to add a copy plugin to the viennacl and viennacl-omp pom.xml s.

        Issue Links

          Activity

          Hide
          Andrew_Palumbo Andrew Palumbo added a comment -

          committeed to master

          Show
          Andrew_Palumbo Andrew Palumbo added a comment - committeed to master
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Mahout-Quality #3451 (See https://builds.apache.org/job/Mahout-Quality/3451/)
          MAHOUT-1955 ViennaCL jars are not being picked up by the shell startup (apalumbo: rev 049619464145feb44805877cea337a4d84179083)

          • (edit) viennacl-omp/pom.xml
          • (edit) viennacl/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Mahout-Quality #3451 (See https://builds.apache.org/job/Mahout-Quality/3451/ ) MAHOUT-1955 ViennaCL jars are not being picked up by the shell startup (apalumbo: rev 049619464145feb44805877cea337a4d84179083) (edit) viennacl-omp/pom.xml (edit) viennacl/pom.xml
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/mahout/pull/294

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/mahout/pull/294
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user andrewpalumbo opened a pull request:

          https://github.com/apache/mahout/pull/294

          MAHOUT-1955: ViennaCL jars are not being picked up by the shell startup script

          As reported by Nikolai Sakharnykh, ViennaCL jars were not being picked up by the shell:

          After building with `mvn clean install -Pviennacl -Phadoop2`

          launching the shell with:

          `MASTER=local[4] mahout spark-shell`

          and loading launching: `scala> timeSparseDRMMMul(1000,1000,1000,1,.02,1234L)`

          ```17/03/15 09:36:17 INFO RootSolverFactory$: Creating org.apache.mahout.viennacl.opencl.GPUMMul solver

          17/03/15 09:36:17 INFO RootSolverFactory$: Unable to create class GPUMMul: attempting OpenMP version

          17/03/15 09:36:17 INFO RootSolverFactory$: Creating org.apache.mahout.viennacl.openmp.OMPMMul solver

          17/03/15 09:36:17 INFO RootSolverFactory$: org.apache.mahout.viennacl.openmp.OMPMMul$

          17/03/15 09:36:17 INFO RootSolverFactory$: Unable to create class OMPMMul: falling back to java version```

          Issue is that `mahout-native-viennacl_2.10` and `mahout-native-viennacl-omp_2.10` were not being copied to the `$MAHOUT_HOME` base dir, and thus not being picked up by `load-shell.sh`.

          By modifying the poms to copy jars to the base dir. This problem is fixed:
          ```
          MASTER=local[4] mahout spark-shell

          {...}
          17/03/16 00:01:27 INFO backend.RootSolverFactory$: Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
          17/03/16 00:01:27 INFO backend.RootSolverFactory$: Successfully created org.apache.mahout.viennacl.opencl.GPUMMul solver
          17/03/16 00:01:27 INFO opencl.GPUMMul$: Using gpuRWCW method
          {...}

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/andrewpalumbo/mahout MAHOUT-1955b

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/mahout/pull/294.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #294


          commit 0e549333bc715e2783c067b2ed34b77ae046edb7
          Author: Andrew Palumbo <apalumbo@apache.org>
          Date: 2017-03-16T06:21:26Z

          MAHOUT-1955 ViennaCL jars are not being picked up by the shell startup script


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user andrewpalumbo opened a pull request: https://github.com/apache/mahout/pull/294 MAHOUT-1955 : ViennaCL jars are not being picked up by the shell startup script As reported by Nikolai Sakharnykh, ViennaCL jars were not being picked up by the shell: After building with `mvn clean install -Pviennacl -Phadoop2` launching the shell with: `MASTER=local [4] mahout spark-shell` and loading launching: `scala> timeSparseDRMMMul(1000,1000,1000,1,.02,1234L)` ```17/03/15 09:36:17 INFO RootSolverFactory$: Creating org.apache.mahout.viennacl.opencl.GPUMMul solver 17/03/15 09:36:17 INFO RootSolverFactory$: Unable to create class GPUMMul: attempting OpenMP version 17/03/15 09:36:17 INFO RootSolverFactory$: Creating org.apache.mahout.viennacl.openmp.OMPMMul solver 17/03/15 09:36:17 INFO RootSolverFactory$: org.apache.mahout.viennacl.openmp.OMPMMul$ 17/03/15 09:36:17 INFO RootSolverFactory$: Unable to create class OMPMMul: falling back to java version``` Issue is that `mahout-native-viennacl_2.10` and `mahout-native-viennacl-omp_2.10` were not being copied to the `$MAHOUT_HOME` base dir, and thus not being picked up by `load-shell.sh`. By modifying the poms to copy jars to the base dir. This problem is fixed: ``` MASTER=local [4] mahout spark-shell {...} 17/03/16 00:01:27 INFO backend.RootSolverFactory$: Creating org.apache.mahout.viennacl.opencl.GPUMMul solver 17/03/16 00:01:27 INFO backend.RootSolverFactory$: Successfully created org.apache.mahout.viennacl.opencl.GPUMMul solver 17/03/16 00:01:27 INFO opencl.GPUMMul$: Using gpuRWCW method {...} You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewpalumbo/mahout MAHOUT-1955 b Alternatively you can review and apply these changes as the patch at: https://github.com/apache/mahout/pull/294.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #294 commit 0e549333bc715e2783c067b2ed34b77ae046edb7 Author: Andrew Palumbo <apalumbo@apache.org> Date: 2017-03-16T06:21:26Z MAHOUT-1955 ViennaCL jars are not being picked up by the shell startup script

            People

            • Assignee:
              Andrew_Palumbo Andrew Palumbo
              Reporter:
              Andrew_Palumbo Andrew Palumbo
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development

                  Agile