Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
0.10.0
-
None
-
None
Description
A'A and possibly A'B can fail with an index out of bounds on the row vector. This seems related to partitioning where some partitions may be empty.
This can be reproduce with the attached data as input into spark-itemsimilarity. This is only A data and the one large csv will complete correctly but passing in the directory of part files will exhibit the error. The data is identical except in the number of files that are used to contain the data.
The error occurs using the local raw filesystem and with master = local and is pretty fast to reach.