[FLINK-2452] Add a playcount threshold to the MusicProfiles example

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.10.0
• Component/s: Library / Graph Processing
• Assignee: Vasia Kalavri
• Reporter: Vasia Kalavri
• Created: 31/Jul/15 16:18
• Updated: 27/Apr/17 10:00
• Resolved: 07/Aug/15 10:04

Description

In the MusicProfiles example, when creating the user-user similarity graph, an edge is created between any 2 users that have listened to the same song (even if once). Depending on the input data, this might produce a projection graph with many more edges than the original user-song graph.
To make this computation more efficient, this issue proposes adding a user-defined parameter that filters out songs that a user has listened to only a few times. Essentially, it is a threshold for playcount, above which a user is considered to like a song.

For reference, with a threshold value of 30, the whole Last.fm dataset is analyzed on my laptop in a few minutes, while no threshold results in a runtime of several hours.

There are many solutions to this problem, but since this is just an example (not a library method), I think that keeping it simple is important.

Thanks to andralungu for spotting the inefficiency!

Issue Links

• links to: GitHub Pull Request #968