Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19759

ALSModel.predict on Dataframes : potential optimization by not using blas

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.1.1
    • 2.3.0
    • ML
    • None

    Description

      In the DataFrame ALS prediction function, we use blas.sdot which may be slower due to the conversion to Arrays. We can try operating on Seqs or another data structure to see if avoiding the conversion makes the operation faster. Ref: https://github.com/apache/spark/pull/17090/files/707bc6b153a7f899fbf3fe2a5675cacba1f95711#diff-be65dd1d6adc53138156641b610fcada

      Attachments

        Activity

          People

            mgaido Marco Gaido
            sueann Sue Ann Hong
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: