[SPARK-43142] DSL expressions fail on attribute with special characters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.5.0
• Component/s: SQL
• Assignee: Willi Raschkowski
• Reporter: Willi Raschkowski
• Created: 14/Apr/23 13:11
• Updated: 25/Apr/23 10:23
• Resolved: 25/Apr/23 10:23

Description

Expressions on implicitly converted attributes fail if the attributes have names containing special characters. They fail even if the attributes are backtick-quoted:

scala> import org.apache.spark.sql.catalyst.dsl.expressions._
import org.apache.spark.sql.catalyst.dsl.expressions._

scala> "`slashed/col`".attr
res0: org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute = 'slashed/col

scala> "`slashed/col`".attr.asc
org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '/' expecting {<EOF>, '.', '-'}(line 1, pos 7)

== SQL ==
slashed/col
-------^^^

Issue Links

• links to: [Github] Pull Request #40794 (rshkv)
• links to: [Github] Pull Request #40902 (rshkv)