[HIVE-25686] UDFSpace result length calculation is incorrect after HADOOP-17901 and HADOOP-17905

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Hive
• Labels: pull-request-available
• Assignee: Mark Bathori
• Reporter: Mark Bathori
• Created: 10/Nov/21 09:00
• Updated: 17/Nov/22 08:48
• Resolved: 16/Nov/21 09:51

Description

The changes added in HADOOP-17901 and HADOOP-17905 introduced some issues in Hive side. UDFSpace is generating false output because the result length is not determined correctly. It is causing unexpected characters in the result.

Repro steps:

create table t(i int);

insert into t values (5),(6),(7),(8),(9),(10),(11),(12),(13),(14);

select i, SPACE(i) from t;

 
Faulty output:

+-----+-----------------+
|  i  |       _c1       |
+-----+-----------------+
| 5   |                 |
| 6   |                 |
| 7   |                |
| 8   |                 |
| 9   |                |
| 10  |               |
| 11  |                 |
| 12  |                |
| 13  |               |
| 14  |              |
+-----+-----------------+

 

Also by using --outputformat=csv the generated file contains 0 ascii characters(“EOF”) instead of spaces.

Issue Links

• is caused by: HADOOP-17901 Performance degradation in Text.append() after HADOOP-16951
• is caused by: HADOOP-17905 Modify Text.ensureCapacity() to efficiently max out the backing array size
• links to: GitHub Pull Request #2786