[FLINK-1474] Cross produces wrong results for non-tiny inputs

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9, 0.8.1
• Component/s: Runtime / Task
• Assignee: Fabian Hueske
• Reporter: Fabian Hueske
• Created: 04/Feb/15 15:04
• Updated: 04/Feb/15 16:26
• Resolved: 04/Feb/15 16:26

Description

The cross operator produces wrong results for larger inputs.
It appears that the number of emitted records is correct, but the values are not.
There seems to be some cut of at 128 elements.

To reproduce add the following test to

{CrossITCase}

and execute it:

	@Test
	public void testCorrectnessOfCrossOfLargeInputs() throws Exception {
		/*
		 * check correctness of cross with larger inputs
		 */

		final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

		DataSet<Long> ds = env.generateSequence(0, 1000);
		DataSet<Long> ds2 = env.generateSequence(0, 1000);
		DataSet<Tuple> crossDs = ds.cross(ds2)
				.filter(new FilterFunction<Tuple2<Long, Long>>() {
					@Override
					public boolean filter(Tuple2<Long, Long> value) throws Exception {
						return value.f0 == value.f1;
					}
				})
				.project(0).sum(0);

		crossDs.writeAsCsv(resultPath);
		env.execute();

		expected = "(500500)\n"; // 500500 = 0+1+2+...+999+1000
	}