[SPARK-43530] Protobuf: Read descriptor file only once at the compile time

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.0
• Component/s: Protobuf
• Epic Link: Protobuf Support
• Assignee: Raghu Angadi
• Reporter: Raghu Angadi
• Created: 16/May/23 22:55
• Updated: 27/May/23 15:19
• Resolved: 27/May/23 15:19

Description

Protobuf functions read from the descriptor file many time (e.g. at each executor). This is unncessary and error prone (e.g. what if the contents change couple of days after the streaming query starts?).

 

It only needs to be read once. 

Issue Links

• links to: [Github] Pull Request #41192 (rangadi)