We would like to add a configuration option that enables a history-server to read files directly from done directory, instead of moving them from intermediate to done directory.
Current setup of mapreduce code, displays files that our moved by a history server from intermediate to done. We cannot have two servers (job cluster and static history server) doing this as it could lead to race conditions.
Our service wants to configure a static history server that displays jobs run on multiple clusters. We would like this server to only read from log directories.