Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12547

Set the default worker threads for compaction to 1

    XMLWordPrintableJSON

Details

    • Wish
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Frontend
    • None
    • ghx-label-13

    Description

      Set the default worker threads for compaction to 1.

      Currently worker threads for compaction to 4, as a result, in the external frontend mode, there are a lot of calls to HMS in idle state. So, it is flooding the catalogD log with unnecessary calls to HMS.

      I0503 14:13:26.944278 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_current_notificationEventId
      I0503 14:13:26.948185 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_config_value
      I0503 14:13:26.949983 15874 CatalogMetastoreServer.java:215] Invoking HMS API: get_next_notification
      I0503 14:13:41.494956 15300 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
      I0503 14:13:41.501145 15303 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
      I0503 14:13:41.507638 15306 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2
      I0503 14:13:41.541407 15309 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2

      After the above-proposed values for the configs, this is the log info in an idle state.

      I1107 15:58:03.130424 533568 CatalogMetastoreServer.java:215] Invoking HMS API: get_next_notification
      I1107 15:58:03.157773 533460 CatalogMetastoreServer.java:215] Invoking HMS API: scheduled_query_poll
      I1107 15:58:03.178020 533480 CatalogMetastoreServer.java:215] Invoking HMS API: find_next_compact2 

       

      Attachments

        Activity

          People

            hemanth619 Sai Hemanth Gantasala
            hemanth619 Sai Hemanth Gantasala
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: