[SPARK-15968] HiveMetastoreCatalog does not correctly validate partitioned metastore relation when searching the internal table cache

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: SQL
• Labels: hive metastore
• Assignee: Michael MacFadden
• Reporter: Michael MacFadden
• Created: 15/Jun/16 16:51
• Updated: 20/Jul/16 16:45
• Resolved: 05/Jul/16 01:50

Description

The getCached method of HiveMetastoreCatalog computes pathsInMetastore from the metastore relation's catalog table. This only returns the table base path, which is not correct for non-empty partitioned tables. As a result, cache lookups on non-empty partitioned tables always miss and these relations are always recomputed.

Issue Links

• links to: [Github] Pull Request #13686 (mallman)
• links to: [Github] Pull Request #13818 (mallman)
• links to: [Github] Pull Request #14064 (yhuai)