Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
2.2.0
Description
org.apache.spark.unsafe.types.UTF8String#compareTo contains the following TODO:
int len = Math.min(numBytes, other.numBytes); // TODO: compare 8 bytes as unsigned long for (int i = 0; i < len; i ++) { // In UTF-8, the byte should be unsigned, so we should compare them as unsigned int.
The todo should be resolved by comparing the maximum number of 64bit words possible in this method, before falling back to unsigned int comparison.