Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3348

Kudu scanner : Avoid per slot check of Vector size in KuduScanner::RelocateValuesFromKudu

    Details

      Description

      Checking the Vector size per column per row in the Kudu scanner consumes 1% of CPU cycles in a basic scan query

      Status KuduScanner::RelocateValuesFromKudu(Tuple* tuple, MemPool* mem_pool) {
        for (int i = 0; i < string_slots_.size(); ++i) {
          const SlotDescriptor* slot = string_slots_[i];
          // NULL handling was done in KuduRowToImpalaTuple.
      
      CPU Time
      1 of 2: 81.2% (6.890s of 8.481s)
      
      impalad ! std::vector<impala::SlotDescriptor*, std::allocator<impala::SlotDescriptor*>>::size - stl_vector.h
      impalad ! impala::KuduScanner::KuduRowToImpalaTuple + 0x80 - kudu-scanner.cc:307
      impalad ! impala::KuduScanner::DecodeRowsIntoRowBatch + 0xd8 - kudu-scanner.cc:236
      impalad ! impala::KuduScanner::GetNext + 0x124 - kudu-scanner.cc:142
      impalad ! impala::KuduScanNode::ScannerThread + 0x3fb - kudu-scan-node.cc:429
      impalad ! boost::function0<void>::operator() + 0x1a - function_template.hpp:767
      impalad ! impala::Thread::SuperviseThread + 0x20c - thread.cc:316
      impalad ! operator()<void (*)(const std::basic_string<char>&, const std::basic_string<char>&, boost::function<void()>, impala::Promise<long int>*), boost::_bi::list0> + 0x5a - bind.hpp:457
      impalad ! boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>::operator() - bind_template.hpp:20
      impalad ! boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run + 0x19 - thread.hpp:116
      impalad ! thread_proxy + 0xd9 - [unknown source file]
      libpthread.so.0 ! start_thread + 0xd0 - [unknown source file]
      libc.so.6 ! clone + 0x6c - [unknown source file]
      

      Todd Lipcon FYI.

        Issue Links

          Activity

          Hide
          mjacobs Matthew Jacobs added a comment -

          This is a trivial fix I already have, but I'd like to wait for some of the test infra work we're starting now to come in before making these changes.

          Show
          mjacobs Matthew Jacobs added a comment - This is a trivial fix I already have, but I'd like to wait for some of the test infra work we're starting now to come in before making these changes.
          Hide
          mjacobs Matthew Jacobs added a comment -

          commit ca3fd401be827680444590fbdf7c1b723434f2c5
          Author: Matthew Jacobs <mj@cloudera.com>
          Date: Sun Jun 12 12:22:23 2016 -0700

          IMPALA-3348: Avoid per-slot check vector size in KuduScanner

          Fixes a small perf issue by avoiding extra calls to check
          a vector size on every slot.

          Testing: Ran EE tests.

          Change-Id: Ie76d33c3d00e3be6d238226d28c4100bb65aac58
          Reviewed-on: http://gerrit.cloudera.org:8080/4688
          Reviewed-by: Matthew Jacobs <mj@cloudera.com>
          Tested-by: Internal Jenkins

          Show
          mjacobs Matthew Jacobs added a comment - commit ca3fd401be827680444590fbdf7c1b723434f2c5 Author: Matthew Jacobs <mj@cloudera.com> Date: Sun Jun 12 12:22:23 2016 -0700 IMPALA-3348 : Avoid per-slot check vector size in KuduScanner Fixes a small perf issue by avoiding extra calls to check a vector size on every slot. Testing: Ran EE tests. Change-Id: Ie76d33c3d00e3be6d238226d28c4100bb65aac58 Reviewed-on: http://gerrit.cloudera.org:8080/4688 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Internal Jenkins

            People

            • Assignee:
              mjacobs Matthew Jacobs
              Reporter:
              mmokhtar Mostafa Mokhtar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development