[SPARK-1801] Open up some private APIs related to creating new RDDs for developers

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: koert kuipers
• Created: 12/May/14 00:44
• Updated: 14/May/14 07:56
• Resolved: 14/May/14 07:56

Description

in writing my own RDD i ran into a few issues with respect to stuff being private in spark.

in compute i would like to return an iterator that respects task killing (as HadoopRDD does), but the mechanics for that are inside the private InterruptibleIterator. also the exception i am supposed to throw (TaskKilledException) is private to spark.

See also:
http://apache-spark-user-list.1001560.n3.nabble.com/Re-writing-my-own-RDD-td5558.html