[FLINK-1989] Sorting of POJO data set from TableEnv yields NotSerializableException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 0.9
• Fix Version/s: 1.0.0
• Component/s: Table SQL / API
• Assignee: Aljoscha Krettek
• Reporter: Fabian Hueske
• Created: 07/May/15 23:02
• Updated: 17/Feb/16 16:43
• Resolved: 17/Feb/16 16:43

Description

Sorting or grouping (or probably any other key operation) on a POJO data set that was created by a TableEnvironment yields a NotSerializableException due to a non-serializable java.lang.reflect.Field object.

I traced the error back to the ExpressionSelectFunction. I guess that a TypeInformation object is stored in the generated user-code function. A PojoTypeInfo holds Field objects, which cannot be serialized.

The following test can be pasted into the SelectITCase and reproduces the problem.

@Test
public void testGroupByAfterTable() throws Exception {

	ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
	TableEnvironment tableEnv = new TableEnvironment();

	DataSet<Tuple3<Integer, Long, String>> ds = CollectionDataSets.get3TupleDataSet(env);

	Table in = tableEnv.toTable(ds, "a,b,c");

	Table result = in
			.select("a, b, c");

	DataSet<ABC> resultSet = tableEnv.toSet(result, ABC.class);

	resultSet
			.sortPartition("a", Order.DESCENDING)
			.writeAsText(resultPath, FileSystem.WriteMode.OVERWRITE);

	env.execute();

	expected = "1,1,Hi\n" + "2,2,Hello\n" + "3,2,Hello world\n" + "4,3,Hello world, " +
			"how are you?\n" + "5,3,I am fine.\n" + "6,3,Luke Skywalker\n" + "7,4," +
			"Comment#1\n" + "8,4,Comment#2\n" + "9,4,Comment#3\n" + "10,4,Comment#4\n" + "11,5," +
			"Comment#5\n" + "12,5,Comment#6\n" + "13,5,Comment#7\n" + "14,5,Comment#8\n" + "15,5," +
			"Comment#9\n" + "16,6,Comment#10\n" + "17,6,Comment#11\n" + "18,6,Comment#12\n" + "19," +
			"6,Comment#13\n" + "20,6,Comment#14\n" + "21,6,Comment#15\n";

}

public static class ABC {
	public int a;
	public long b;
	public String c;
}