Description
while use split function in sql, it behave differently between 2.3 and 3.2, which cause incorrect problem.
we can use this sql to reproduce this problem:
create table split_test ( id int,name string)
insert into split_test values(1,"abc;def")
explain extended select split(name,'\\\\;') from split_test
spark3:
spark-sql> Explain extended select split(name,'\\\\;') from split_test;
== Parsed Logical Plan ==
'Project [unresolvedalias('split('name, \\;), None)]
+- 'UnresolvedRelation [split_test], [], false
spark2:
spark-sql> Explain extended select split(name,'\\\\;') from split_test;
== Parsed Logical Plan ==
'Project [unresolvedalias('split('name, \;), None)]
+- 'UnresolvedRelation split_test
It looks like the deal of escape is different