Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Currently, the BlobServer uses a local storage and, in addition when the HA mode is set, a distributed file system, e.g. hdfs. This, however, is only used by the JobManager and all TaskManager instances request blobs from the JobManager. By using the distributed file system there as well, we would lower the load on the JobManager and increase scalability.