Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.0.1
Description
The Murmur3 and xxHash64 implementations in Spark do not produce the correct results on big-endian systems. This causes test failures on my target platform (s390x).
These hash functions require that multi-byte chunks be interpreted as integers encoded in little-endian byte order. This requires byte reversal when using multi-byte unsafe operations on big-endian platforms.
I have a PR ready for discussion and review.