Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6816

Statestore spends a lot of time in GetMinSubscriberTopicVersion()

    XMLWordPrintableJSON

    Details

      Description

      Samples: 13K of event 'cycles', Event count (approx.): 1200870513
        20.23%      statestored  impalad              [.] impala::Statestore::GetMinSubscriberTopicVersion(std::string const&, std::string*)
         7.68%      statestored  [kernel.kallsyms]    [k] find_busiest_group
         3.46%      statestored  impalad              [.] impala::Statestore::Subscriber::LastTopicVersionProcessed(std::string const&) const
         3.26%      statestored  libc-2.12.so         [.] __memcmp_sse4_1
         1.41%      statestored  [kernel.kallsyms]    [k] find_next_bit
         1.40%      statestored  [kernel.kallsyms]    [k] cpumask_next_and
         1.21%      statestored  libpthread-2.12.so   [.] pthread_mutex_lock
         1.04%      statestored  libc-2.12.so         [.] memcpy
         1.01%      statestored  [kernel.kallsyms]    [k] _spin_lock
         0.98%      statestored  impalad              [.] 0x000000000088f903
         0.93%      statestored  impalad              [.] 0x000000000088f8f5
         0.91%      statestored  impalad              [.] 0x000000000088f8ea
         0.85%      statestored  [kernel.kallsyms]    [k] ixgbe_xmit_frame_ring
         0.77%      statestored  impalad              [.] 0x000000000088f8e3
         0.75%      statestored  impalad              [.] 0x000000000088f900
         0.75%      statestored  impalad              [.] impala::Statestore::IsPrioritizedTopic(std::string const&)
         0.73%      statestored  impalad              [.] 0x000000000088f8fa
         0.72%      statestored  impalad              [.] operator new[](unsigned long)
         0.68%      statestored  [kernel.kallsyms]    [k] tcp_recvmsg
         0.67%      statestored  impalad              [.] 0x000000000088f8fd
         0.66%      statestored  impalad              [.] impala::Statestore::Topic::BuildDelta(std::string const&, long, impala::TTopicDelta*)
         0.61%      statestored  [kernel.kallsyms]    [k] thread_return
         0.60%      statestored  impalad              [.] 0x000000000088f8f2
         0.60%      statestored  libstdc++.so.6       [.] std::string::compare(std::string const&) const
         0.59%      statestored  impalad              [.] 0x000000000088f8e6
         0.56%      statestored  impalad              [.] 0x000000000088f8ee
         0.56%      statestored  libcrypto.so.1.0.1e  [.] aesni_encrypt
         0.55%      statestored  impalad              [.] 0x000000000088f8e0
         0.55%      statestored  [kernel.kallsyms]    [k] tcp_transmit_skb
         0.53%      statestored  [kernel.kallsyms]    [k] fget_light
         0.51%      statestored  impalad              [.] std::_Rb_tree<void*, std::pair<void* const, std::shared_ptr<impala::ThriftClientImpl> >, std::_Select1st<std::pair<void* const, std::shared_ptr<impala::ThriftCl
         0.50%      statestored  impalad              [.] apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport, apache::thrift::transport::TBufferBase>::readAll_virt(unsigned char*
         0.50%      statestored  impalad              [.] impala::Statestore::DoSubscriberUpdate(impala::Statestore::UpdateKind, int, impala::Statestore::ScheduledSubscriberUpdate const&)
         0.49%      statestored  libssl.so.1.0.1e     [.] tls1_enc
         0.48%      statestored  libssl.so.1.0.1e     [.] ssl3_read_bytes
      

      We are spending most of our time computing this for non-catalog topics, where it's not even used.

      There are a couple of ways we could fix this that I can think of:

      • Avoid including this information for topics where we're not interested in it
      • Cache or precompute the value somehow to avoid iterating over all subscribers every time

        Attachments

          Activity

            People

            • Assignee:
              tarmstrong Tim Armstrong
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: