[SPARK-23740] Add FPGrowth Param for filtering out very common items

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.0
• Component/s: ML
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Joseph K. Bradley
• Created: 19/Mar/18 21:11
• Updated: 08/Oct/19 05:41
• Resolved: 08/Oct/19 05:41

Description

It would be handy to have a Param in FPGrowth for filtering out very common items. This is from a use case where the dataset had items appearing in 99.9%+ of the rows. These common items were useless, but they caused the algorithm to generate many unnecessary itemsets. Filtering useless common items beforehand can make the algorithm much faster.

Issue Links

• relates to: SPARK-7211 Improvements for FPGrowth