[FLINK-1628] Strange behavior of "where" function during a join

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9
• Fix Version/s: 0.9, 0.8.2
• Component/s: API / DataSet
• Labels: batch
• Assignee: Fabian Hueske
• Reporter: Daniel Bali
• Created: 02/Mar/15 17:52
• Updated: 28/Feb/19 14:30
• Resolved: 10/Mar/15 00:51

Description

Hello!

If I use the `where` function with a field list during a join, it exhibits strange behavior.

Here is the sample code that triggers the error: https://gist.github.com/balidani/d9789b713e559d867d5c

This example joins a DataSet with itself, then counts the number of rows. If I use `.where(0, 1)` the result is (22), which is not correct. If I use `EdgeKeySelector`, I get the correct result (101).

When I pass a field list to the `equalTo` function (but not `where`), everything works again.

If I don't include the `groupBy` and `reduceGroup` parts, everything works.

Also, when working with large DataSets, passing a field list to `where` makes it incredibly slow, even though I don't see any exceptions in the log (in DEBUG mode).

Does anybody know what might cause this problem?

Thanks!