How do a I create a Set valued column in Spark?

Frank Wilson Source

In the Spark Shell I am trying to make a Set valued column, but Spark interprets the Set as an array.

scala> var df=List((Set(1,2),1,2), (Set(2,1),2,1)).toDF("s","a","b")
scala> df.show()
+------+---+---+
|     s|  a|  b|
+------+---+---+
|[1, 2]|  1|  2|
|[2, 1]|  2|  1|
+------+---+---+
scala> df.select("s")
res02: String = [s: array<int>]

How can I have a Set valued column in spark?

scalaapache-spark

Answers

comments powered by Disqus