Details
-
Sub-task
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
None
Description
The current creation of NetworkEnvironment is relying on IOManager from TaskManagerServices. In order not to rely on external specific components for implementing shuffle environment, and let the specific implementation creates internal components if required.
The current abstract IOManager has two roles, one is for file channel management based on temp directories configuration, and the other is providing abstract methods for reading/writing files. We could further extract the file channel management as a separate internal class which could be reused for all the required components, like current BoundedBlockingSubpartition. To do so, the shuffle environment should also creates its internal channel manager to break dependency with IOManager.
Attachments
Issue Links
- links to
