Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10691

Impala crashes sporadically when multiple CAST(FORMAT)

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 3.4.0
    • Impala 4.0.0
    • Backend
    • ghx-label-12

    Description

      One way to repro. This crashes Impala after a few runs.

      use functional;
      set max_scan_range_length=16384;
      set mt_dop=2;
      select count(*) from alltypes a1, alltypes a2 where cast(cast(a1.date_string_col as date format 'MM/dd/yy') as string format 'MM/dd/yy') = cast(cast(a2.date_string_col as date format 'MM/dd/yy') as string format 'MM/dd/yy');
      

      The trace can be different but this is one frequently coming up:

      F0430 12:32:43.029990  6427 datetime-iso-sql-format-tokenizer.cc:138] 614cc7aa439af2d5:96e9c4c200000002] Check failed: str_begin <= *current_pos && *current_pos < str_end
      *** Check failure stack trace: ***
          @          0x548af0c  google::LogMessage::Fail()
          @          0x548c7fc  google::LogMessage::SendToLog()
          @          0x548a86a  google::LogMessage::Flush()
          @          0x548e468  google::LogMessageFatal::~LogMessageFatal()
          @          0x24d619c  impala::datetime_parse_util::IsoSqlFormatTokenizer::ProcessNextToken()
          @          0x24d5d9e  impala::datetime_parse_util::IsoSqlFormatTokenizer::Tokenize()
          @          0x2cce85c  impala::CastFormatExpr::OpenEvaluator()
          @          0x2c4e43f  impala::ScalarExpr::OpenEvaluator()
          @          0x2c93690  impala::ScalarFnCall::OpenEvaluator()
          @          0x2cce5b9  impala::CastFormatExpr::OpenEvaluator()
          @          0x2c55959  impala::ScalarExprEvaluator::Open()
          @          0x2c559e8  impala::ScalarExprEvaluator::Open()
          @          0x2b9acb8  impala::HashTableCtx::Open()
          @          0x2ad8725  impala::PartitionedHashJoinNode::Open()
          @          0x2bc1c86  impala::AggregationNode::Open()
          @          0x24a61df  impala::FragmentInstanceState::Open()
          @          0x24a27ea  impala::FragmentInstanceState::Exec()
          @          0x23e65af  impala::QueryState::ExecFInstance()
      

      Another possible trace is:

      #7  0x000000000239d0a1 in impala::DateParser::Format[abi:cxx11](impala::datetime_parse_util::DateTimeFormatContext const&, impala::DateValue const&) (dt_ctx=..., date=...)
          at /home/boroknagyz/Impala/be/src/runtime/date-parse-util.cc:135
      #8  0x000000000227a5d7 in impala::DateValue::Format[abi:cxx11](impala::datetime_parse_util::DateTimeFormatContext const&) const (this=0x7f21674d268c, dt_ctx=...)
          at /home/boroknagyz/Impala/be/src/runtime/date-value.cc:152
      #9  0x0000000002cccdd2 in impala::CastFunctions::CastToStringVal (ctx=0x93bcee8, val=...) at /home/boroknagyz/Impala/be/src/exprs/cast-functions-ir.cc:219
      #10 0x0000000002cab261 in impala::ScalarFnCall::InterpretEval<impala_udf::StringVal> (this=0xcf29c00, eval=0xaa8a1a0, row=0xd08a000) at /home/boroknagyz/Impala/be/src/exprs/scalar-fn-call.cc:490
      #11 0x0000000002c90a7d in impala::ScalarFnCall::GetStringValInterpreted (this=0xcf29c00, eval=0xaa8a1a0, row=0xd08a000) at /home/boroknagyz/Impala/be/src/exprs/scalar-fn-call.cc:556
      #12 0x0000000002c5c330 in impala::ScalarExpr::GetStringVal (this=0xcf29c00, eval=0xaa8a1a0, row=0xd08a000) at /home/boroknagyz/Impala/be/src/exprs/scalar-expr.inline.h:62
      #13 0x0000000002c595a7 in impala::ScalarExprEvaluator::GetValue (this=0xaa8a1a0, expr=..., row=0xd08a000) at /home/boroknagyz/Impala/be/src/exprs/scalar-expr-evaluator.cc:304
      #14 0x0000000002c59281 in impala::ScalarExprEvaluator::GetValue (this=0xaa8a1a0, row=0xd08a000) at /home/boroknagyz/Impala/be/src/exprs/scalar-expr-evaluator.cc:255
      #15 0x00000000023d9798 in impala::KrpcDataStreamSender::HashRow (this=0x1363f200, row=0xd08a000) at /home/boroknagyz/Impala/be/src/runtime/krpc-data-stream-sender.cc:977
      #16 0x00000000023e3cfc in impala::KrpcDataStreamSender::HashAndAddRows (this=0x1363f200, batch=0xaa8a0d0) at /home/boroknagyz/Impala/be/src/runtime/krpc-data-stream-sender-ir.cc:38
      

      There is another scenario that gives an error incorrectly:

      [localhost:21050] functional> select count(*) from alltypes a1, alltypes a2 where cast(cast(a1.date_string_col as date format 'MM/dd/yy') as string format 'yyyyMMdd') = cast(cast(a2.date_string_col as date format 'MM/dd/yy') as string format 'yyyyMMdd');
      ...
      ERROR: UDF ERROR: String to Date parse failed. Input '07/03/10' doesn't match with format 'yyyyMMdd'
      

      A TSAN build shows data race pointing here:

      https://github.com/apache/impala/blob/bca1b43efb72d839962f9e6999ca747fca7f17d9/be/src/exprs/cast-format-expr.cc#L37

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            csringhofer Csaba Ringhofer
            gaborkaszab Gabor Kaszab
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment