Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3434

Distributed block matrix

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: MLlib
    • Labels:

      Description

      This JIRA is for discussing distributed matrices stored in block sub-matrices. The main challenge is the partitioning scheme to allow adding linear algebra operations in the future, e.g.:

      1. matrix multiplication
      2. matrix factorization (QR, LU, ...)

      Let's discuss the partitioning and storage and how they fit into the above use cases.

      Questions:

      1. Should it be backed by a single RDD that contains all of the sub-matrices or many RDDs with each contains only one sub-matrix?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shivaram Shivaram Venkataraman
                Reporter:
                mengxr Xiangrui Meng
              • Votes:
                7 Vote for this issue
                Watchers:
                36 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: