Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14707

Linear algebra: clarify light vs heavy constructors and accessors

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ML
    • Labels:

      Description

      MLlib linear algebra provides methods for constructing Vectors and Matrices and for accessing the vector/matrix data. There are currently 2 types of these constructors and accessors:

      • light: avoid data copy and validation, useful for converting between MLlib types and numpy, Breeze, etc.
      • heavy: copy or validate data, useful for constructing MLlib types from user inputs

      MLlib is not very consistent about these and does not document which ops are light vs. heavy. This JIRA is for:

      • First, discussing which ops should be light vs heavy to choose a consistent API
      • Next, creating subtasks for Scala and Python for updating the implementations and clarifying the docs

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                josephkb Joseph K. Bradley
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: