Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4278

Don't abort Catalog startup quickly if HMS is not present

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      If the catalog daemon can't contact the HMS on startup, it will fail out of the Catalog constructor in MetaStoreClientPool.addClients().

      We might consider not doing so, but instead retry for a longer time to allow the catalog and the HMS to be started concurrently.

        Activity

        Show
        bharathv bharath v added a comment - After the switch to RetryingHiveMetastoreClient in commit [1] , I think we can tune hive.metastore.failure.retries [2,3] [1] https://github.com/cloudera/Impala/commit/1ec0c47fdb2b42c95f99051f5e83e0e86ff0c1fe [2] https://github.com/cloudera/hive/blob/cdh5-1.1.0_5.8.0/metastore/src/java/org/apache/hadoop/hive/metastore/RetryingMetaStoreClient.java#L61 [3] https://github.com/cloudera/hive/blob/cdh5-1.1.0_5.8.0/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L383
        Hide
        henryr Henry Robinson added a comment -

        I don't want to do that for all connection attempts - if the HMS fails, we need users to get feedback quickly (rather than have long retry times). During start-up it's a different story, because the HMS may not have failed, but just be starting concurrently.

        Show
        henryr Henry Robinson added a comment - I don't want to do that for all connection attempts - if the HMS fails, we need users to get feedback quickly (rather than have long retry times). During start-up it's a different story, because the HMS may not have failed, but just be starting concurrently.
        Show
        henryr Henry Robinson added a comment - Fixed in https://github.com/apache/incubator-impala/commit/60414f06330aa62d06a67821309c8ff452de5a5f

          People

          • Assignee:
            attilaj Attila Jeges
            Reporter:
            henryr Henry Robinson
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development