Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5524

NullPointerException during planning with DISABLE_UNSAFE_SPILLS=TRUE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-2

      Description

      Setting the DISABLE_UNSAFE_SPILLS query option to TRUE leads to a NullPointerException during query planning if all tables of the query have stats.

      Repro:

      create table t (i int);
      compute stats t;
      set disable_unsafe_spills=true;
      select * from t;
      ERROR: NullPointerException: null
      

      Stack:

      I0615 23:06:54.581377 11228 jni-util.cc:176] java.lang.NullPointerException
      	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1027)
      	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1104)
      	at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156)
      

      The problem is here:

          boolean disableSpilling =
              queryCtx.client_request.query_options.isDisable_unsafe_spills()
                && !queryCtx.tables_missing_stats.isEmpty() <-- tables_missing_stats can be null
                && !analysisResult.getAnalyzer().hasPlanHints();
          queryCtx.setDisable_spilling(disableSpilling);
      

      The reason why "tables_missing_stats" is null can be found in Frontend.createPlanExecInfo():

      ...
          // Clear pre-existing lists to avoid adding duplicate entries in FE tests.
          queryCtx.unsetTables_missing_stats(); <-- sets to null
          queryCtx.unsetTables_with_corrupt_stats();
          for (TTableName tableName: tablesMissingStats) {
            queryCtx.addToTables_missing_stats(tableName);
          }
          for (TTableName tableName: tablesWithCorruptStats) {
            queryCtx.addToTables_with_corrupt_stats(tableName);
          }
          for (TTableName tableName: tablesWithMissingDiskIds) {
            queryCtx.addToTables_missing_diskids(tableName);
          }
      ...
      

        Attachments

          Activity

            People

            • Assignee:
              thundergun Vincent Tran
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: