[SYSTEMDS-2059] Improve parfor optimization and initialization overhead

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemML 1.1
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 06/Jan/18 00:03
• Updated: 12/Jan/18 00:26
• Resolved: 12/Jan/18 00:26

Description

In this scripts that use parfor in functions and call these functions 1000s of times, the parfor optimization and initialization overhead can be substantial. This task aims to improve this overhead, by parallelizing the worker initialization of local workers (which also include deep copies of update in place variables) and generally removing unnecessary optimization overhead.