Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.92.1, 0.94.0
-
None
-
Reviewed
Description
Currently each coprocessor is loaded with a URLClassLoader that puts the coprocessor's jar at the beginning of the classpath. The URLClassLoader always tries to load classes from the parent ClassLoader first and only attempts to load from its own configured URLs if the class was not found by the parent. This class loading behavior can be problematic for coprocessors that have common dependencies with HBase but whose versions are incompatible. For example, I have a coprocessor that depends on a different version of Avro than the version used by HBase. The current class loading behavior results in NoSuchMethodErrors in my coprocessor because some Avro classes have already been loaded by HBase, and the ClassLoader for my coprocessor picks up HBase's loaded classes first.
My proposed solution to this problem is to use a custom ClassLoader when instantiating coprocessor instances. This custom ClassLoader would always attempt to load classes from the coprocessor's jar first and would only delegate to the parent ClassLoader if the class were not found in the coprocessor jar. However, certain classes would need to be exempt from this behavior. As an example, if the Copcoessor interface were loaded by both the region server's ClassLoader and the coprocessor's custom ClassLoader, then the region server would get a ClassCastException when attempting to cast the coprocessor instance to the Coprocessor interface. This problem can be avoided by defining a set of class name prefixes that would be exempt from loading by the custom ClassLoader. When loading a class, if the class starts with any of these prefixes (e.g. "org.apache.hadoop"), then the ClassLoader would delegate immediately to the parent ClassLoader.
I've already implemented a patch to provide this functionality which I'll attach shortly.