[HIVE-9492] Enable caching in MapInput for Spark

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: spark-branch
• Component/s: Spark
• Assignee: Jimmy Xiang
• Reporter: Xuefu Zhang
• Created: 28/Jan/15 05:19
• Updated: 21/Oct/22 07:31

Description

Because of the IOContext problem (HIVE-8920, HIVE-9084), RDD caching is currently disabled in MapInput. Prototyping shows that the problem can solved. Thus, we should formalize the prototype and enable the caching. A good query to test this is:

from (select * from dec union all select * from dec2) s
insert overwrite table dec3 select s.name, sum(s.value) group by s.name
insert overwrite table dec4 select s.name, s.value order by s.value;

Issue Links

• relates to: SPARK-2688 Need a way to run multiple data pipeline concurrently