Current log-aggregation implementation supports logaggregation to default filecontext only. This patch is to support logaggregation to any of the supported filesystems within hadoop eco-system (hdfs, s3, swiftfs etc). So for example a customer could use hdfs as default filesystem but use s3 or swiftfs for logaggregation. Current implementation makes mixed usages of FileContext+AbstractFileSystem apis as well as FileSystem apis which is confusing.
This patch does two things:
- moves logaggregation implementation to use only FileContext apis
- adds support for doing log aggregation on non-default filesystem as well.
- changes TestLogAggregationService to use local filesystem itself instead of mocking the behavior