[SPARK-47904] Preserve case in Avro schema when using enableStableIdentifiersForUnionType

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0, 3.5.2
• Fix Version/s: 4.0.0, 3.5.2
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Ivan Sadikov
• Reporter: Ivan Sadikov
• Created: 18/Apr/24 22:37
• Updated: 22/Apr/24 22:36
• Resolved: 22/Apr/24 22:36

Description

When enableStableIdentifiersForUnionType is enabled, all of the types are lowercased which creates a problem when field types are case-sensitive: 

Schema.createEnum("myENUM", "", null, List[String]("E1", "e2").asJava),
Schema.createRecord("myRecord2", "", null, false, List[Schema.Field](new Schema.Field("F", Schema.create(Type.FLOAT))).asJava)

 would become

struct<member_myenum: string, member_myrecord2: struct<f: float>>  

but instead should be 

struct<member_myENUM: string, member_myRecord2: struct<F: float>>  

 

Issue Links

• links to: GitHub Pull Request #46126
• links to: GitHub Pull Request #46169