Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3418

[MLlib] Additional BLAS and Local Sparse Matrix support

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • MLlib
    • None

    Description

      Currently MLlib doesn't have Level-2 and Level-3 BLAS support. For Multi-Model training, adding support for Level-3 BLAS functions is vital.

      In addition, as most real data is sparse, support for Local Sparse Matrices will also be added, as supporting sparse matrices will save a lot of memory and will lead to better performance. The ability to left multiply a dense matrix with a sparse matrix, i.e. `C := alpha * A * B + beta * C` where `A` is a sparse matrix will also be added. However, `B` and `C` will remain as Dense Matrices for now.

      I will post performance comparisons with other libraries that support sparse matrices such as Breeze and Matrix-toolkits-JAVA (MTJ) in the comments.

      Attachments

        Activity

          People

            brkyvz Burak Yavuz
            brkyvz Burak Yavuz
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: