[SPARK-38664] Support compact EventLog when there are illegal characters in the path

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 2.4.8, 3.2.1
• Component/s: Deploy
• Assignee: Unassigned
• Reporter: w l
• Created: 26/Mar/22 07:42
• Updated: 05/Apr/22 02:26
• Resolved: 05/Apr/22 02:26

Description

When a path has illegal characters, it is escaped.  The file will not be found and compactor  will not work.
 
The path:  "/tmp/app%"
Exception:
java.io.FileNotFoundException: /tmp/app%25/events_7_app.compact.inprogress (No such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.(FileOutputStream.java:213)
at java.io.FileOutputStream.(FileOutputStream.java:101)
at org.apache.spark.deploy.history.EventLogFileWriter.initLogFile(EventLogFileWriters.scala:95)
at org.apache.spark.deploy.history.SingleEventLogFileWriter.start(EventLogFileWriters.scala:223)
at org.apache.spark.deploy.history.EventLogFileCompactor.rewrite(EventLogFileCompactor.scala:152)
at org.apache.spark.deploy.history.EventLogFileCompactor.compact(EventLogFileCompactor.scala:94)

Issue Links

• links to: [Github] Pull Request #35979 (lw33)