Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8790

IllegalStateException: Illegal reference to non-materialized slot

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 3.1.0, Impala 3.2.0
    • Fix Version/s: Impala 3.3.0
    • Component/s: Frontend
    • Labels:
      None

      Description

      Reproduce:

      $ hdfs dfs -put foo.parq hdfs:///tmp
      impala> create table foo (uid string, cid string) stored as parquet;
      impala> load data inpath 'hdfs:///tmp/foo.parq' into table foo;
      

      With the stats, the following query hits an IllegalStateException:

      impala> compute stats foo;
      impala> explain select uid, cid,
         rank() over (partition by uid order by count(*) desc)
      from (select uid, cid from foo) w
      group by uid, cid;
      
      ERROR: IllegalStateException: Illegal reference to non-materialized slot: tid=1 sid=2

      Without the stats, it runs successfully:

      impala> drop stats foo;
      impala> explain select uid, cid,
         rank() over (partition by uid order by count(*) desc)
      from (select uid, cid from foo) w
      group by uid, cid;
      +------------------------------------------------------------------------------------+
      | Explain String                                                                     |
      +------------------------------------------------------------------------------------+
      | Max Per-Host Resource Reservation: Memory=84.02MB Threads=5                        |
      | Per-Host Resource Estimates: Memory=304MB                                          |
      | WARNING: The following tables are missing relevant table and/or column statistics. |
      | common_action.foo                                                                  |
      |                                                                                    |
      | PLAN-ROOT SINK                                                                     |
      | |                                                                                  |
      | 07:EXCHANGE [UNPARTITIONED]                                                        |
      | |                                                                                  |
      | 03:ANALYTIC                                                                        |
      | |  functions: rank()                                                               |
      | |  partition by: uid                                                               |
      | |  order by: count(*) DESC                                                         |
      | |  window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW                       |
      | |  row-size=40B cardinality=1.10K                                                  |
      | |                                                                                  |
      | 02:SORT                                                                            |
      | |  order by: uid ASC NULLS FIRST, count(*) DESC                                    |
      | |  row-size=32B cardinality=1.10K                                                  |
      | |                                                                                  |
      | 06:EXCHANGE [HASH(uid)]                                                            |
      | |                                                                                  |
      | 05:AGGREGATE [FINALIZE]                                                            |
      | |  output: count:merge(*)                                                          |
      | |  group by: uid, cid                                                              |
      | |  row-size=32B cardinality=1.10K                                                  |
      | |                                                                                  |
      | 04:EXCHANGE [HASH(uid,cid)]                                                        |
      | |                                                                                  |
      | 01:AGGREGATE [STREAMING]                                                           |
      | |  output: count(*)                                                                |
      | |  group by: uid, cid                                                              |
      | |  row-size=32B cardinality=1.10K                                                  |
      | |                                                                                  |
      | 00:SCAN HDFS [common_action.foo]                                                   |
      |    HDFS partitions=1/1 files=1 size=5.19KB                                         |
      |    row-size=24B cardinality=1.10K                                                  |
      +------------------------------------------------------------------------------------+
      Fetched 37 row(s) in 0.03s
      

       

        Attachments

        1. foo.parq
          5 kB
          Quanlong Huang

          Issue Links

            Activity

              People

              • Assignee:
                stiga-huang Quanlong Huang
                Reporter:
                stiga-huang Quanlong Huang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: