Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Won't Fix
-
1.3.1
-
None
-
None
Description
Let's say we have an RDD of Array[Double] where zero values are explictly recorded. Ie (0.0, 0.0, 3.2, 0.0...) If we want to transform this into an RDD of sparse vectors, we currently have to:
arr_doubles.map{ array =>
val indexElem: Seq[(Int, Double)] = array.zipWithIndex.filter(tuple => tuple._1 != 0.0).map(tuple => (tuple._2, tuple._1))
Vectors.sparse(arrray.length, indexElem)
}
Notice that there is a map step at the end to switch the order of the index and the element value after .zipWithIndex. There should be a factory method on the Vectors class that allows you to avoid this flipping of tuple elements when using zipWithIndex.
