[FLINK-2253] ALS fails if the data flow is split up and the input is non-deterministic

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Component/s: Library / Machine Learning
• Labels: ML
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 19/Jun/15 17:14
• Updated: 28/Feb/19 22:57
• Resolved: 28/Feb/19 22:57

Description

If the temporaryPath of Flink's ALS implementation is set, then the ALS job is executed one part after the other. This also includes the routing information calculation for the user and item blocks. If the input DataSet is calculated non-deterministically, e.g. if it's the result of a first operation, then the resulting routing information won't be consistent and ALS fails.

A solution would be to pin the input DataSet so that it will only be executed once. Until we have this functionality, I propose to simply execute the user and item routing information at the same time.