[SPARK-6814] Support sorting for any data type in SparkR

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Won't Fix
• Component/s: SparkR
• External issue URL: https://sparkr.atlassian.net/browse/SPARKR-201
• Assignee: Unassigned
• Reporter: Shivaram Venkataraman
• Created: 09/Apr/15 20:04
• Updated: 19/Jun/16 21:17
• Resolved: 19/Jun/16 21:17

Description

I get various "return status == 0 is false" and "unimplemented type" errors trying to get data out of any rdd with top() or collect(). The errors are not consistent. I think spark is installed properly because some operations do work. I apologize if I'm missing something easy or not providing the right diagnostic info – I'm new to SparkR, and this seems to be the only resource for SparkR issues.
Some logs:

Browse[1]> top(estep.rdd, 1L)
Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) : 
  unimplemented type 'list' in 'orderVector1'
Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
Execution halted
15/02/13 19:11:57 ERROR Executor: Exception in task 0.0 in stage 14.0 (TID 14)
org.apache.spark.SparkException: R computation failed with
 Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) : 
  unimplemented type 'list' in 'orderVector1'
Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
Execution halted
	at edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:69)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
	at org.apache.spark.scheduler.Task.run(Task.scala:54)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
15/02/13 19:11:57 WARN TaskSetManager: Lost task 0.0 in stage 14.0 (TID 14, localhost): org.apache.spark.SparkException: R computation failed with
 Error in order(unlist(part, recursive = FALSE), decreasing = !ascending) : 
  unimplemented type 'list' in 'orderVector1'
Calls: do.call ... Reduce -> <Anonymous> -> func -> FUN -> FUN -> order
Execution halted
        edu.berkeley.cs.amplab.sparkr.BaseRRDD.compute(RRDD.scala:69)
        org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:262)
        org.apache.spark.rdd.RDD.iterator(RDD.scala:229)
        org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
        org.apache.spark.scheduler.Task.run(Task.scala:54)
        org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        java.lang.Thread.run(Thread.java:745)