[HIVE-28530] Set files in thread safe manner in HiveSequenceFileInputFormat

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 4.1.0
• Component/s: HiveServer2
• Security Level: Public (Viewable by anyone)
• Labels: pull-request-available
• Assignee: Sourabh Badhya
• Reporter: Xiaomin Zhang
• Created: 19/Sep/24 14:21
• Updated: 05/Nov/24 06:24
• Resolved: 25/Sep/24 16:10

Description

When running Hive load tests, we observed Beeline can fetch wrong query result which is from another one running at same time. We ruled out Load Balancing issue, because it happened to a single HiveServer2. And we found this issue only happens when hive.query.result.cached.enabled is false.

All test queries are in the same format as below:

select concat('total record (test_$PID)=',count(*)) as count_record from t1t

We randomized the query by replacing the $PID with the Beeline PID and the test driver ran 10 Beeline concurrently. The table t1t is static and has a few rows. So now the test driver can check if the query result is equal to: total record (test_recon_mock_$PID)=2

When query result cache is disabled, we can see randomly query got a wrong result, and can always reproduced. For example, below two queries were running in parallel:

queryId=hive_20240701103742_ff1adb2d-e9eb-448d-990e-00ab371e9db6): select concat('total record (test_21535)=',count(*)) as count_record from t1t

queryId=hive_20240701103742_9bdfff92-89e1-4bcd-88ea-bf73ba5fd93d): select concat('total record (test_21566)=',count(*)) as count_record from t1t

While the second query is supposed to get below result:
total record (test_21566)=2

But actually Beeline got below result:
total record (test_21535)=2

There is no error in the HS2 log.

Issue Links

• is caused by: HIVE-21279 Avoid moving/rename operation in FileSink op for SELECT queries
• relates to: HIVE-28609 HiveSequenceFileInputFormat should be cloned or not be cached
• links to: GitHub Pull Request #5461