Minor Description
The following SQL query will cause an infinite loop in SqlSerializer2 code:
val df = sqlContext.sql("select null where 1 = 1") df.unionAll(df).sort("_c0").collect()
The same problem occurs if we add more null-literals, but does not occur as long as there is a column of any other type (e.g. select 1, null where 1 == 1).
I think that what's happening here is that if you have a row that consists only of columns of NullType (not columns of other types which happen to only contain null values, but only columns of null literals), SqlSerializer will end up writing / reading no data for rows of this type. Since the deserialization stream will never try to read any data but nevertheless will be able to return an empty row, DeserializationStream.asIterator will go into an infinite loop since there will never be a read to trigger an EOF exception.
Attachments
Issue Links
- links to
