Resolution: Won't Fix
Affects Version/s: Impala 1.4
Fix Version/s: None
Environment:Linux 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
According to http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/ciiu_math_functions.html
the fnv_hash UDF implements the 64 bits FNV-1a variation.
According to http://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function
the algorithm should be seeded with the 64-bit FNV offset basis value: 14695981039346656037 (in hex, 0xcbf29ce484222325)
Implementing this, I did not obtain the same FNV 1a hashes as Impala
E.g. with impala-shell I obtain
whereas it should be -6615550055289275125
By looking at the Impala unit tests:
I see that the algorithm is seeded with the 32 bits offset basis
instead of FNV64_SEED.
If I update my algorithm and seed it with the 32 bits offset basis, I obtain the same hashes as impala.
For backward compatibility, it may not be easy to fix. Or it could be deprecated and replaced with a fixed UDF ?