Cosine, Euclidean, Manhattan distance calculation in Sparklyr

ROY Source

I am using below configuration for Sparklyr:

sparklyr = 0.7.0
spark = 2.2.1

Now, I have two matrices as sdf (m1, m2) and want to calculate below distances/similarity measures:

  1. Cosine
  2. Euclidean
  3. Manhattan

I know how to do this in "R" i.e. using "proxy" library:

library(proxy)
m4 = simil(m1,m2,method = "cosine",by_rows = TRUE)

m5 = dist(m1,m2,method = "Euclidean",by_rows = TRUE)

m6 = dist(m1,m2,method = "Manhattan",by_rows = TRUE)

Kindly provide solution on how to compute this in sparklyr.

Thanks.

rsparklyr

Answers

comments powered by Disqus