Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2923

Implement some basic linalg operations in MLlib

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0
    • MLlib
    • None

    Description

      We use breeze for linear algebra operations. Breeze operations are user-friendly but there are some concerns:

      1. creating temp objects, e.g., `val z = a * x + b * y`
      2. multi-method is not used in some operators, e.g., `axpy`. If we pass in SparseVector as a generic Vector, it will use activeIterator, which is slow
      3. calling native BLAS if it is available, which might not be good for level-1 methods

      Having some basic BLAS operations implemented in MLlib can help simplify the current implementation and improve some performance.

      Attachments

        Activity

          People

            mengxr Xiangrui Meng
            mengxr Xiangrui Meng
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: