Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.4.0
-
Reviewed
Description
This is a continuation of HADOOP-17901.
Right now we use a factor of 1.5x to increase the byte array if it's full. However, if the size reaches a certain point, the increment is only (current size + length). This can cause performance issues if the textual data which we intend to store is beyond this point.
Instead, let's max out the array to the maximum. Based on different sources, a safe choice seems to be Integer.MAX_VALUE - 8 (see ArrayList, AbstractCollection, HashTable, etc).
Attachments
Issue Links
- causes
-
HIVE-25686 UDFSpace result length calculation is incorrect after HADOOP-17901 and HADOOP-17905
-
- Closed
-
- Dependent
-
HADOOP-17901 Performance degradation in Text.append() after HADOOP-16951
-
- Resolved
-
- links to