[SPARK-19771] Support OR-AND amplification in Locality Sensitive Hashing (LSH)

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.1.0
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Yun Ni
• Created: 28/Feb/17 19:14
• Updated: 21/May/19 04:15
• Resolved: 21/May/19 04:15

Description

The current LSH implementation only supports AND-OR amplification. We need to discuss the following questions before we goes to implementations:
(1) Whether we should support OR-AND amplification
(2) What API changes we need for OR-AND amplification
(3) How we fix the approxNearestNeighbor and approxSimilarityJoin internally.

Issue Links

• is related to: SPARK-18392 LSH API, algorithm, and documentation follow-ups
• is related to: SPARK-18450 Add AND-amplification to Locality Sensitive Hashing