How to find the salary between two ranges. Here is my spark sql dataFrame code for finding the salary between 10000 to 20000
empData.where($"Salary" > 10000 && $"Salary" < 20000).orderBy($"Salary".desc).show()
I believe have alternative solution to use between. How can i use between approach?scalaapache-sparkapache-spark-sql
If you wish to use between, you can use sparkSQL and run logic as query. For e.g.
empData.createOrReplaceTempView("empDataTempTable") val filteredData = spark.sql("select * from empDataTempTable where salary between 10000 and 20000 order by salary desc") filteredData.show()
For random lookups in a column and filter process, sparkSQL and DataFrame gives almost same performance results.
I believe have alternative solution to use between.
Just like that: