[HIVE-22161] UDF: FunctionRegistry synchronizes on org.apache.hadoop.hive.ql.udf.UDFType class

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.2, 3.1.2, 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: UDF
• Labels: concurrency performance
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 30/Aug/19 06:53
• Updated: 17/Nov/22 08:50
• Resolved: 04/Sep/19 00:22

Description

There's a hidden synchronization across threads when looking up isStateful and isDeterministic.

https://github.com/apache/hive/blob/master/common/src/java/org/apache/hive/common/util/AnnotationUtils.java#L27

  // to avoid https://bugs.openjdk.java.net/browse/JDK-7122142
  public static <T extends Annotation> T getAnnotation(Class<?> clazz, Class<T> annotationClass) {
    synchronized (annotationClass) {
      return clazz.getAnnotation(annotationClass);
    }
  }

This is serializing multiple threads initializing UDFs (or checking them during compilation) & also being locked across threads for each instance of GenericUDFOpEqual in the specific scenario.

https://bugs.openjdk.java.net/browse/JDK-7122142 is fixed in jdk8+