[SYSTEMDS-3158] Ultra-sparse matrix generation creates biased values

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: SystemDS 2.2
• Assignee: Matthias Boehm
• Reporter: Matthias Boehm
• Created: 11/Oct/21 18:28
• Updated: 11/Oct/21 21:45
• Resolved: 11/Oct/21 21:45

Description

Relative to the blocksize, if generated matrices get very sparse, we see a systematic bias toward the upper end of the value range.

X =rand(rows=10000000,cols=100000,min =0,max =100,sparsity=0.00000001 )
write(X, "tmp/sparse.txt", format="text")

After closer investigation, this was due to correlated nnz and value random number generators (same seed). We now add a prime to the second random number generator, which fixed the problem