Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5432

SetMemLimitExceeded DCHECK no longer valid

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Backend
    • Labels:
      None
    • Epic Color:
      ghx-label-7

      Description

      DCHECK(query_status_.IsMemLimitExceeded()) fails due to conditional setting of MemLimitExceeded status. The query_status is only set to MemLimitExceeded if the prior status was ok (query_status_.ok).

      void RuntimeState::SetMemLimitExceeded(MemTracker* tracker,
          int64_t failed_allocation_size, const ErrorMsg* msg) {
        Status status = tracker->MemLimitExceeded(this, msg == nullptr ? "" : msg->msg(),
            failed_allocation_size);
        {
          lock_guard<SpinLock> l(query_status_lock_);
          if (query_status_.ok()) query_status_ = status;
        }
        LogError(status.msg());
        // Add warning about missing stats except for compute stats child queries.
        if (!query_ctx().__isset.parent_query_id &&
            query_ctx().__isset.tables_missing_stats &&
            !query_ctx().tables_missing_stats.empty()) {
          LogError(ErrorMsg(TErrorCode::GENERAL,
              GetTablesMissingStatsWarning(query_ctx().tables_missing_stats)));
        }
        DCHECK(query_status_.IsMemLimitExceeded());
      }
      

      This was discovered while running: ./tests/run-tests.py query_test/test_queries.py -k test_very_large_strings
      which caused some impalad instances to hit the DCHECK and generate minidumps in my environment (and fail the test).

      Removing the DCHECK still doesn't allow the test to pass due to what I think is better memory error reporting, but I'm still investigating the failures to better understand what is going on.

        Attachments

          Activity

            People

            • Assignee:
              jfs John Sherman
              Reporter:
              jfs John Sherman
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: