[SPARK-973] Mark fields in RDD class that are not used in workers as @transient to reduce task size

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.8.1, 0.9.0
• Assignee: Reynold Xin
• Reporter: Reynold Xin
• Created: 02/Dec/13 16:04
• Updated: 07/Dec/13 14:10
• Resolved: 07/Dec/13 14:10

Description

I can see a few candidates to mark as transient.

  /** Optionally overridden by subclasses to specify how they are partitioned. */
  val partitioner: Option[Partitioner] = None

  /** A friendly name for this RDD */
  var name: String = null

  /** User-defined generator of this RDD*/
  var generator = Utils.getCallSiteInfo.firstUserClass