Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22161

UDF: FunctionRegistry synchronizes on org.apache.hadoop.hive.ql.udf.UDFType class

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.2, 3.1.2, 4.0.0
    • Fix Version/s: 4.0.0
    • Component/s: UDF

      Description

      There's a hidden synchronization across threads when looking up isStateful and isDeterministic.

      https://github.com/apache/hive/blob/master/common/src/java/org/apache/hive/common/util/AnnotationUtils.java#L27

        // to avoid https://bugs.openjdk.java.net/browse/JDK-7122142
        public static <T extends Annotation> T getAnnotation(Class<?> clazz, Class<T> annotationClass) {
          synchronized (annotationClass) {
            return clazz.getAnnotation(annotationClass);
          }
        }
      

      This is serializing multiple threads initializing UDFs (or checking them during compilation) & also being locked across threads for each instance of GenericUDFOpEqual in the specific scenario.

      https://bugs.openjdk.java.net/browse/JDK-7122142 is fixed in jdk8+

        Attachments

        1. HIVE-22161.1.patch
          1 kB
          Gopal Vijayaraghavan

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              gopalv Gopal Vijayaraghavan Assign to me
              Reporter:
              gopalv Gopal Vijayaraghavan

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment