[HIVE-27138] MapJoinOperator throws NPE when computing OuterJoin with filter expressions on small table

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Labels: hive-4.0.0-must pull-request-available
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 14/Mar/23 09:27
• Updated: 29/Mar/24 17:49
• Resolved: 15/Sep/23 09:26

Description

Hive throws NPE when running mapjoin_filter_on_outerjoin.q using Tez engine. (I used TestMiniLlapCliDriver.)
The NPE is thrown by CommonJoinOperator.getFilterTag(), which just retreives the last object from the given list.
To the best of my knowledge, if Hive selects MapJoin to perform Join operation, filterTag should be computed and appended to a row before the row is passed to MapJoinOperator.
In the case of MapReduce engine, this is done by HashTableSinkOperator.
However, I cannot find any logic pareparing filterTag for small tables when Hive uses Tez engine.

I think there are 2 available options:
1. Don't use MapJoinOperator if a small table has filter expression.
2. Add a new logic that computes and passes filterTag to MapJoinOperator.

I am working on the second option and ready to discuss about it.
It would be grateful if you could give any opinion about this issue.

Issue Links

• links to: GitHub Pull Request #4115