Hive
  1. Hive
  2. HIVE-6672

JARs loaded by permanent functions don't work properly with HiveServer2

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: HiveServer2, UDF
    • Labels:
      None

      Description

      Permanent functions which specify JAR files to load do not seem to work properly with HiveServer2. While the function can be created and see with SHOW FUNCTIONS, doing DESCRIBE FUNCTION or using the UDF in a query results in java.lang.ClassNotFoundException.

      It looks like the JAR is only added to the classloader of the very first thread which references the UDF. Subsequent threads will have a different class loader which may not have loaded the JAR, but because the UDF is already in the FunctionRegistry these threads do not attempt to load the resources for the UDF.

        Issue Links

          Activity

          Jason Dere created issue -
          Hide
          Jason Dere added a comment -

          Patch v1 which addresses this issue. When lookup up the FunctionInfo for a permanent UDF from the FunctionRegistry, check if the classloader can resolve the UDF class, and if that fails then load the JARs necessary for that permanent UDF.

          Show
          Jason Dere added a comment - Patch v1 which addresses this issue. When lookup up the FunctionInfo for a permanent UDF from the FunctionRegistry, check if the classloader can resolve the UDF class, and if that fails then load the JARs necessary for that permanent UDF.
          Jason Dere made changes -
          Field Original Value New Value
          Attachment HIVE-6672.1.patch [ 12635692 ]
          Jason Dere made changes -
          Assignee Jason Dere [ jdere ]
          Jason Dere made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Jason Dere added a comment -
          Show
          Jason Dere added a comment - RB at https://reviews.apache.org/r/19435/
          Jason Dere made changes -
          Remote Link This issue links to "https://reviews.apache.org/r/19435/ (Web Link)" [ 14631 ]
          Hide
          Jason Dere added a comment -

          Harish Butani Is this a bug that would be able to get into Hive-0.13?

          Show
          Jason Dere added a comment - Harish Butani Is this a bug that would be able to get into Hive-0.13?
          Hide
          Ashutosh Chauhan added a comment -

          Patch looks good. I wonder if we can write a test case for this, that will be good.

          Show
          Ashutosh Chauhan added a comment - Patch looks good. I wonder if we can write a test case for this, that will be good.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12635692/HIVE-6672.1.patch

          SUCCESS: +1 5440 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1912/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1912/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12635692

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12635692/HIVE-6672.1.patch SUCCESS: +1 5440 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1912/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1912/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12635692
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk & 0.13. Thanks, Jason!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk & 0.13. Thanks, Jason!
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.13.0 [ 12324986 ]
          Resolution Fixed [ 1 ]
          Nemon Lou made changes -
          Link This issue relates to HIVE-9095 [ HIVE-9095 ]
          Yongzhi Chen made changes -
          Link This issue is related to HIVE-10453 [ HIVE-10453 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5d 4h 37m 1 Jason Dere 20/Mar/14 00:08
          Patch Available Patch Available Resolved Resolved
          3d 16h 53m 1 Ashutosh Chauhan 23/Mar/14 17:01

            People

            • Assignee:
              Jason Dere
              Reporter:
              Jason Dere
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development