[HIVE-12995] LLAP: Synthetic file ids need collision checks

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: llap
• Labels: TODOC2.1
• Assignee: Sergey Shelukhin
• Reporter: Gopal Vijayaraghavan
• Created: 03/Feb/16 22:42
• Updated: 21/Jun/16 15:44
• Resolved: 15/Mar/16 20:51

Description

LLAP synthetic file ids do not have any way of checking whether a collision occurs other than a data-error.

Synthetic file-ids have only been used with unit tests so far - but they will be needed to add cache mechanisms to non-HDFS filesystems.

In case of Synthetic file-ids, it is recommended that we track the full-tuple (path, mtime, len) in the cache so that a cache-hit for the synthetic file-id can be compared against the parameters & only accepted if those match.

Issue Links

• is related to: HIVE-13225 LLAP: Stripe metadata cache holds unreachable keys
• links to: RB