[SPARK-26850] Make EventLoggingListener LOG_FILE_PERMISSIONS configurable

• Type: Wish
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.2.3, 2.3.2, 2.4.0
• Component/s: Scheduler, Spark Core
• Assignee: Unassigned
• Reporter: Hua Zhang
• Created: 08/Feb/19 14:16
• Updated: 17/May/20 17:48
• Resolved: 05/Mar/19 08:19

Description

private[spark] object EventLoggingListener extends Logging

{ ... private val LOG_FILE_PERMISSIONS = new FsPermission(Integer.parseInt("770", 8).toShort) ... }

 

Currently the event log files are hard-coded with permission 770.

It would be fine if this permission is configurable.

User case: The spark application is submitted by user A but the spark history server is started by user B. Currently user B cannot access the history event files created by user A. When permission is set to 775, this will be possible.