Type: New Feature
Affects Version/s: None
Fix Version/s: None
Component/s: Query Processor
There have been several requests on the mailing list for
information about how to access the DistributedCache from UDFs, e.g.:
While responses to these emails suggested several workarounds, the only correct
way of accessing the distributed cache is via the static methods of Hadoop's
DistributedCache class, and all of these methods require that the JobConf be passed
in as a parameter. Hence, giving UDFs access to the distributed cache
reduces to giving UDFs access to the JobConf.
I propose the following changes to GenericUDF/UDAF/UDTF:
- Add an exec_init(Configuration conf) method that is called during Operator initialization at runtime.
- Change the name of the "initialize" method to "compile_init" to make it clear that this method is called at compile-time.