Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1799

AsyncKuduClient.discoverTablets has very expensive logging

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:

      Description

      Impala finds that the following logging is causing extreme contention in concurrent planning of queries containing tables w/ a large number of partitions.

            LOG.info("Learned about tablet {} for table '{}' with partition {}",
                     rt.getTabletId(), tableName, rt.getPartition());
      

      This stack dominates the cost of planning:

      org.apache.log4j.Category.callAppenders(LoggingEvent)	165,411	6,692,346,585,303
         org.apache.log4j.Category.forcedLog(String, Priority, Object, Throwable)	165,411	6,692,346,585,303
            org.apache.log4j.Category.log(String, Priority, Object, Throwable)	165,411	6,692,346,585,303
               org.slf4j.impl.Log4jLoggerAdapter.info(String, Object[])	165,411	6,692,346,585,303
                  org.apache.kudu.client.AsyncKuduClient.discoverTablets(KuduTable, byte[], List, long)	165,411	6,692,346,585,303
                     org.apache.kudu.client.AsyncKuduClient$MasterLookupCB.call(Master$GetTableLocationsResponsePB)	165,411	6,692,346,585,303
                        org.apache.kudu.client.AsyncKuduClient$MasterLookupCB.call(Object)	165,411	6,692,346,585,303
                           com.stumbleupon.async.Deferred.doCall(Callback)	165,411	6,692,346,585,303
      

        Attachments

          Activity

            People

            • Assignee:
              danburkert Dan Burkert
              Reporter:
              mjacobs Matthew Jacobs
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: