Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1837

Sparse/Dense Matrix analysis for Matrix Multiplication

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: Math
    • Labels:
      None

      Description

      In matrix multiplication, Sparse Matrices can easily turn dense and bloat memory, one fully dense column and one fully dense row can cause a sparse %*% sparse operation have a dense result.

      There are two issues here one with a quick Fix and one a bit more involved:

      1. in ABt.Scala use check the `MatrixFlavor` of the combiner and use the flavor of the Block as the resulting Sparse or Dense matrix type:
        val comb = if (block.getFlavor == MatrixFlavor.SPARSELIKE) {
                      new SparseMatrix(prodNCol, block.nrow).t
                    } else {
                      new DenseMatrix(prodNCol, block.nrow).t
                    }
        

        a simlar check needs to be made in the blockify transformation.

      1. More importantly, and more involved is to do an actual analysis of the resulting matrix data in the in-core mmul class and use a matrix of the appropriate Structure as a result.

        Attachments

        1. compareDensityTest.ods
          86 kB
          Andrew Palumbo

          Activity

            People

            • Assignee:
              Andrew_Palumbo Andrew Palumbo
              Reporter:
              Andrew_Palumbo Andrew Palumbo
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: