Apache Spark with Scala - Add rows from a matrix

italktothewind Source

I have a CoordinateMatrix object in Scala. I would like to add, for instance, row 1 and row 2, and store the result in row 1. What is the optimal way to do that? Thanks in advance.



answered 1 year ago ashburshui #1

Let me help you to illustrate common ideas about it.

first of all, the internal core data structure of CoordinateMatrix:

case class MatrixEntry(i: Long, j: Long, value: Double)

the CoordinateMatrix is just a wrapper of RDD[MatrixEntry] with some easy to use interfaces.

so what you can do it with this class is:

constructor: new CoordinateMatrix(entries: RDD[MatrixEntry])

so you can use (and even rehashing) the merged rdd to provide a new contruction of it.

comments powered by Disqus