Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13160

HS2 unable to load UDFs on startup when HMS is not ready

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.1
    • Fix Version/s: 2.1.0
    • Component/s: HiveServer2
    • Labels:
      None

      Description

      The error looks like this:

      2016-02-18 14:43:54,251 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 14:48:54,692 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 14:48:54,692 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 14:48:55,692 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 14:53:55,800 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 14:53:55,800 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 14:53:56,801 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 14:58:56,967 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 14:58:56,967 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 14:58:57,994 WARN  hive.ql.metadata.Hive: [main]: Failed to register all functions.
      java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
              at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1492)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)
              at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2915)
      .......
      016-02-18 14:58:57,997 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 15:03:58,094 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 15:03:58,095 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 15:03:59,095 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 15:08:59,203 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 15:08:59,203 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 15:09:00,203 INFO  hive.metastore: [main]: Trying to connect to metastore with URI thrift://host-10-17-81-201.coe.cloudera.com:9083
      2016-02-18 15:14:00,304 WARN  hive.metastore: [main]: Failed to connect to the MetaStore Server...
      2016-02-18 15:14:00,304 INFO  hive.metastore: [main]: Waiting 1 seconds before next connection attempt.
      2016-02-18 15:14:01,306 INFO  org.apache.hive.service.server.HiveServer2: [main]: Shutting down HiveServer2
      2016-02-18 15:14:01,308 INFO  org.apache.hive.service.server.HiveServer2: [main]: Exception caught when calling stop of HiveServer2 before retrying start
      java.lang.NullPointerException
              at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:283)
              at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:351)
              at org.apache.hive.service.server.HiveServer2.access$400(HiveServer2.java:69)
              at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:545)
      

      And then none of the functions will be available for use as HS2 does not re-register them after HMS is up and ready.

      This is not desired behaviour, we shouldn't allow HS2 to be in a servicing state if function list is not ready. Or, maybe instead of initialize the function list when HS2 starts, try to load the function list when each Hive session is created. Of course we can have a cache of function list somewhere for better performance, but we would better decouple it from class Hive.

        Attachments

        1. HIVE-13160.1.patch
          1 kB
          Aihua Xu
        2. HIVE-13160.2.patch
          1 kB
          Aihua Xu

          Activity

            People

            • Assignee:
              aihuaxu Aihua Xu
              Reporter:
              ericlin Eric Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: