[SPARK-4823] rowSimilarities

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Auto Closed
• Component/s: MLlib
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Reza Zadeh
• Created: 11/Dec/14 02:15
• Updated: 06/Jun/19 13:57
• Resolved: 06/Jun/19 13:57

Description

RowMatrix has a columnSimilarities method to find cosine similarities between columns.

A rowSimilarities method would be useful to find similarities between rows.

This is JIRA is to investigate which algorithms are suitable for such a method, better than brute-forcing it. Note that when there are many rows (> 10^6), it is unlikely that brute-force will be feasible, since the output will be of order 10^12.

Issue Links

• relates to: SPARK-3066 Support recommendAll in matrix factorization model
• relates to: SPARK-4675 Find similar products and similar users in MatrixFactorizationModel
• links to: [Github] Pull Request #6213 (debasish83)