[SYSTEMDS-3031] Improve SPOOF CUDA rowwise handling of intermediate memory

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: SystemDS 2.2
• Component/s: NativeRuntime
• Labels: CUDA GPU codegen
• Epic Link: spoof-cuda
• Assignee: Mark Dokter
• Reporter: Mark Dokter
• Created: 17/Jun/21 09:32
• Updated: 17/Jun/21 09:32

Description

The allocation of temporary memory for SPOOF CUDA rowwise operators is happening in native space and requires rows*columns*num_intermediates*sizeof(usedDataType) bytes of memory at the moment to enable all threadblocks to work concurrently.

This improvement should

• move the allocation to the Java side of SystemDS to integrate better with the gpu memory manager
• reduce the memory footprint by either having less thread blocks process more rows or queueing less blocks at once for execution.
• update the estimate of intermediate memory at HOP level