Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19759

ALSModel.predict on Dataframes : potential optimization by not using blas

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.3.0
    • Component/s: ML
    • Labels:
      None

      Description

      In the DataFrame ALS prediction function, we use blas.sdot which may be slower due to the conversion to Arrays. We can try operating on Seqs or another data structure to see if avoiding the conversion makes the operation faster. Ref: https://github.com/apache/spark/pull/17090/files/707bc6b153a7f899fbf3fe2a5675cacba1f95711#diff-be65dd1d6adc53138156641b610fcada

        Attachments

          Activity

            People

            • Assignee:
              mgaido Marco Gaido
              Reporter:
              sueann Sue Ann Hong
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: