Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2743

Propagate IS NOT NULL to scans from cols in inner joins

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.2, Impala 2.3.0
    • Fix Version/s: None
    • Component/s: Frontend

      Description

      Columns involved inner joins and filters should have a "IS NOT NULL" filter pushed to the ScanNode. Other operators reading the value for the column shouldn't be re-checking if the value is null again.

      When using Parquet file format ideally we should be using the stats to short circuit these filters.

      Currently whenever a slot is read by any of the operators SlotRef checks if the value is NULL which adds unnecessary branches and instructions.

      BigIntVal SlotRef::GetBigIntVal(ExprContext* context, TupleRow* row) {
        DCHECK_EQ(type_.type, TYPE_BIGINT);
        Tuple* t = row->GetTuple(tuple_idx_);
        if (t == NULL || t->IsNull(null_indicator_offset_)) return BigIntVal::null();
        return BigIntVal(*reinterpret_cast<int64_t*>(t->GetSlot(slot_offset_)));
      }
      
      impalad!impala::Tuple::IsNull
      impalad!impala::SlotRef::GetBigIntVal - [Unknown]
      impalad!impala::SlotRef::GetBigIntVal - [Unknown]
      impalad!impala::ExprContext::GetValue+0x13b - [Unknown]:[Unknown]
      impalad!impala::HashTableCtx::EvalRow+0xb7 - [Unknown]:[Unknown]
      impalad!impala::PartitionedHashJoinNode::Partition::BuildHashTableInternal<(bool)0>+0x372 - [Unknown]:[Unknown]
      impalad!impala::PartitionedHashJoinNode::Partition::BuildHashTable+0xd - [Unknown]:[Unknown]
      impalad!impala::PartitionedHashJoinNode::BuildHashTables+0xa0 - [Unknown]:[Unknown]
      impalad!impala::PartitionedHashJoinNode::ProcessBuildInput+0xc71 - [Unknown]:[Unknown]
      impalad!impala::PartitionedHashJoinNode::ConstructBuildSide+0x106 - [Unknown]:[Unknown]
      impalad!impala::BlockingJoinNode::BuildSideThread+0x7f - [Unknown]:[Unknown]
      impalad!impala::Thread::SuperviseThread+0x1b9 - [Unknown]:[Unknown]
      impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void (void)>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void (void)>>, boost::_bi::value<impala::Promise<long>*>>>>::run+0x7f - [Unknown]:[Unknown]
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mmokhtar Mostafa Mokhtar
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: