Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21045

Add HMS total api count stats and connection pool stats to metrics

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0, 3.2.0
    • Component/s: Standalone Metastore
    • Labels:
      None

      Description

      There are two key metrics which I think we lack and which would be really great to help with scaling visibility in HMS.

      Total API calls duration stats
      We already compute and log the duration of API calls in the PerfLogger. We don't have any gauge or timer on what the average duration of an API call is for the past some bucket of time. This will give us an insight into if there is load on the server which is increasing the average API response time.
       
      Connection Pool stats
      We can use different connection pooling libraries such as bonecp or hikaricp. These pool managers expose statistics such as average time waiting to get a connection, number of connections active, etc. We should expose this as a metric so that we can track if the the connection pool size configured is too small and we are saturating!

      These metrics would help catch problems with HMS resource contention before they actually have jobs failing.

        Attachments

        1. HIVE-21045.1.patch
          26 kB
          Karthik Manamcheri
        2. HIVE-21045.2.branch-3.patch
          23 kB
          Karthik Manamcheri
        3. HIVE-21045.2.patch
          26 kB
          Karthik Manamcheri
        4. HIVE-21045.3.patch
          26 kB
          Karthik Manamcheri
        5. HIVE-21045.4.patch
          26 kB
          Karthik Manamcheri
        6. HIVE-21045.5.patch
          26 kB
          Karthik Manamcheri
        7. HIVE-21045.6.patch
          26 kB
          Karthik Manamcheri
        8. HIVE-21045.7.patch
          26 kB
          Karthik Manamcheri
        9. HIVE-21045.branch-3.patch
          23 kB
          Karthik Manamcheri

          Issue Links

            Activity

              People

              • Assignee:
                karthik.manamcheri Karthik Manamcheri
                Reporter:
                karthik.manamcheri Karthik Manamcheri
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: