[SYSTEMDS-2672] Range Indexing (all rows, columns) recompile

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: SystemDS 3.2
• Component/s: Compiler
• Labels: MinorProject Optimization
• Assignee: Matthias Boehm
• Reporter: Olga Ovcharenko
• Created: 24/Sep/20 23:36
• Updated: 03/Jul/23 15:51
• Resolved: 03/Jul/23 12:17

Description

Federated range indexing is not optimized while accessing all rows and columns. 

A = federated(addresses=list($in), ranges=list(list(0, 0), list($rows, $cols)))
R = A[,]

ncol, nrow variables instead of dim.

------FED fedinit matrix.SCALAR.STRING.true _Var2.LIST.UNKNOWN.false _Var5.LIST.UNKNOWN.false _mVar6.MATRIX.FP64 
------CP rmvar _Var2 _Var5 
------CP nrow _mVar6.MATRIX.FP64.false _Var7.SCALAR.INT64 
------CP ncol _mVar6.MATRIX.FP64.false _Var8.SCALAR.INT64 
------CP createvar _mVar9 target/testTemp/functions/federated/FederatedRightIndexInFedTest/FederatedRightIndexInFedTest/scratch_space//t0/temp3 true MATRIX binary -1 -1 -1 -1 copy 
------CP rightIndex _mVar6.MATRIX.FP64 1.SCALAR.INT64.true _Var7.SCALAR.INT64.false 1.SCALAR.INT64.true _Var8.SCALAR.INT64.false _mVar9.MATRIX.FP64

 
NOTE!:

This is not related to federated, but in general a property we want to support. The reason it shows as a federated issue originally is because it would pull back the matrix from Federated to local.
We want to remove all calls to slice out the entire block in the compiler in any case be it Federated or not.

Such that code like

A = read("x.csv")
B = A[,]

Acts in the same way as

A = read("x.csv")
B = A

Where there is no actual copy of the matrix A in memory but simply an extra variable in the execution, that is compiled away by other optimizations.