Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9887

ASAN builds timeout frequently

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Impala 4.0.0
    • Impala 4.0.0
    • None

    Description

      It has happened atleast couple of times in this week on the ASAN builds. The custom cluster tests fails with the test setup error and logs suggest that coordinator nodes don't start up due to the following exception trace:

      F0623 17:22:42.725920 25786 frontend.cc:136] IllegalStateException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
      CAUSED BY: RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
      CAUSED BY: InvocationTargetException: null
      CAUSED BY: MetaException: Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
              at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:631)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:241)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
              at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:112)
              at org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
              at org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:78)
              at org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
              at org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:163)
              at org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:155)
              at org.apache.impala.service.Frontend.<init>(Frontend.java:331)
              at org.apache.impala.service.Frontend.<init>(Frontend.java:288)
              at org.apache.impala.service.JniFrontend.<init>(JniFrontend.java:144)
      Caused by: java.net.ConnectException: Connection refused (Connection refused)
              at java.net.PlainSocketImpl.socketConnect(Native Method)
              at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
              at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
              at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
              at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
              at java.net.Socket.connect(Socket.java:589)
              at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
              ... 19 more
      
      . Impalad exiting.
      

      Attachments

        Activity

          People

            joemcdonnell Joe McDonnell
            vihangk1 Vihang Karajgaonkar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: