Currently, EventLoggingListener will generate application logs using, in the worst case, five different entries in the file system:
- The directory to hold the files
- One file for the Spark version
- One file for the event logs
- One file to identify the compression codec of the event logs
- One file to say the application is finished.
It would be better to be more friendly to the NameNodes and use a single entry for all of those.