[SPARK-34108] Cache lookup doesn't work in certain cases

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.1.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Chao Sun
• Created: 14/Jan/21 03:13
• Updated: 28/Jan/21 05:40
• Resolved: 28/Jan/21 05:40

Description

Currently, caching a temporary or permenant view doesn't work in certain cases. For instance, in the following:

CREATE TABLE t (key bigint, value string) USING parquet
CREATE VIEW v1 AS SELECT key FROM t
CACHE TABLE v1
SELECT key FROM t

The last SELECT query will hit the cached v1. On the other hand:

CREATE TABLE t (key bigint, value string) USING parquet
CREATE VIEW v1 AS SELECT key FROM t ORDER by key
CACHE TABLE v1
SELECT key FROM t ORDER BY key

The SELECT won't hit the cache.

It seems this is related to EliminateView. In the second case, it will insert an extra project operator which makes the comparison on canonicalized plan during cache lookup fail.

Issue Links

• is duplicated by: SPARK-34269 simplify view resolution
• is related to: SPARK-33647 cache table not working for persisted view
• links to: [Github] Pull Request #31182 (sunchao)
• links to: [Github] Pull Request #31182 (sunchao)