[FLINK-9167] Approximate KNN with Incremental Insertion

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Won't Do
• Component/s: Library / Machine Learning
• Assignee: Unassigned
• Reporter: Alex Klibisz
• Created: 13/Apr/18 04:28
• Updated: 28/Feb/19 22:57
• Resolved: 28/Feb/19 22:57

Description

I'm new to Flink, and I'm curious about an extension of approximate KNN to support  incremental insertion to the index.

I'm specifically interested in this use case: You build an index from a training set of vectors. As your application runs, you ingest a stream of new vectors (e.g. users posting new content). For every new vector, you compute its neighbors against the existing index. Then you immediately insert the new vector to the index such that it can be returned for subsequent queries.

Perhaps this is possible with current components of Flink, or maybe another streaming tool already has a comparable  implementation? If so, I would appreciate any pointers or links to examples.

If it's not available, is there interest in implementing such a feature? If so, I would be interested in making an attempt.

I appreciate any tips or insight. Thanks!