[SPARK-19289] UnCache Dataset using Name

• Type: Wish
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 2.1.0
• Component/s: Spark Core, SQL
• Labels: features
• Flags: Important
• External issue ID: Related to - https://issues.apache.org/jira/browse/SPARK-8480
• Assignee: Unassigned
• Reporter: Kaushal Prajapati
• Created: 19/Jan/17 13:44
• Updated: 23/Jan/17 10:50
• Resolved: 23/Jan/17 10:50

Description

We can Cache and Uncache any table using its name in Spark Sql.

df.createTempView("myTable")
sqlContext.cacheTable("myTable")
sqlContext.uncacheTable("myTable")

Likewise if it is possible to have some kind of uniqueness for names in DataSets and an abstraction like the same that we have for tables. It would be very useful

scala> val df = sc.range(1,1000).toDF
df: org.apache.spark.sql.DataFrame = [value: bigint]

scala> df.setName("MyDataset")
res0: df.type = MyDataset

scala> df.cache
res1: df.type = MyDataset

sqlContext.getDataSet("MyDataset")
sqlContext.uncacheDataSet("MyDataset")