Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5145

CTAS failing when creating from a view with error "Unsupported type 'null_type'"

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.7.0, Impala 2.8.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:

      Description

      There is an apparent regression in Impala 2.7. The following works in Impala 2.6; but fails in Impala 2.7 and up. Note : students1 is an arbitrary table.

      [root@host-10-17-80-137 ~]# impala-shell
      Starting Impala Shell without Kerberos authentication
      Connected to host-10-17-80-137.coe.cloudera.com:21000
      Server version: impalad version 2.7.0-cdh5.10.0 RELEASE (build 785a073cd07e2540d521ecebb8b38161ccbd2aa2)
      ***********************************************************************************
      Welcome to the Impala shell.
      (Impala Shell v2.7.0-cdh5.10.0 (785a073) built on Fri Jan 20 12:03:56 PST 2017)
      
      [host-10-17-80-137.coe.cloudera.com:21000] > create view v1_nul_tst as select cast(null as string) col1 from students1;
      Query: create view v1_nul_tst as select cast(null as string) col1 from students1
      Query submitted at: 2017-03-27 15:17:51 (Coordinator: http://host-10-17-80-137:25000)
      Query progress can be monitored at: http://host-10-17-80-137:25000/query_plan?query_id=ed4ef0e6b343d1a9:ad0527c600000000
      
      Fetched 0 row(s) in 0.13s
      [host-10-17-80-137.coe.cloudera.com:21000] > create table t1_nul_tst as select * from v1_nul_tst where 1=2;
      Query: create table t1_nul_tst as select * from v1_nul_tst where 1=2
      Query submitted at: 2017-03-27 15:18:17 (Coordinator: http://host-10-17-80-137:25000)
      ERROR: AnalysisException: Unsupported type 'null_type' in column 'col1' of table 't1_nul_tst'
      CAUSED BY: TableLoadingException: Unsupported type 'null_type' in column 'col1' of table 't1_nul_tst'
      
      [host-10-17-80-137.coe.cloudera.com:21000] > drop view v1_nul_tst;
      Query: drop view v1_nul_tst
      [host-10-17-80-137.coe.cloudera.com:21000] > create view v1_nul_tst as select cast(null as varchar(10)) col1 from students1;
      Query: create view v1_nul_tst as select cast(null as varchar(10)) col1 from students1
      Query submitted at: 2017-03-27 15:21:10 (Coordinator: http://host-10-17-80-137:25000)
      Query progress can be monitored at: http://host-10-17-80-137:25000/query_plan?query_id=be4c7f476c73e6d8:2f25ac1100000000
      
      Fetched 0 row(s) in 0.12s
      
      [host-10-17-80-137.coe.cloudera.com:21000] > desc v1_nul_tst;
      Query: describe v1_nul_tst
      +------+-------------+---------+
      | name | type | comment |
      +------+-------------+---------+
      | col1 | varchar(10) | |
      +------+-------------+---------+
      Fetched 1 row(s) in 3.76s
      [host-10-17-80-137.coe.cloudera.com:21000] > create table t1_nul_tst as select * from v1_nul_tst where 1=2;
      Query: create table t1_nul_tst as select * from v1_nul_tst where 1=2
      Query submitted at: 2017-03-27 15:21:55 (Coordinator: http://host-10-17-80-137:25000)
      ERROR: AnalysisException: Unsupported type 'null_type' in column 'col1' of table 't1_nul_tst'
      CAUSED BY: TableLoadingException: Unsupported type 'null_type' in column 'col1' of table 't1_nul_tst'
      

        Activity

        Hide
        bgitline Boris Gitline added a comment -

        The workaround is: use

        set enable_expr_rewrites=false;

        in front of the CTAS.

        Show
        bgitline Boris Gitline added a comment - The workaround is: use set enable_expr_rewrites=false; in front of the CTAS.
        Hide
        alex.behm Alexander Behm added a comment -

        The simplest solution is to avoid constant folding "CAST(NULL AS TYPE)" expressions specifically. The problem with those is that with our current approach it's very difficult to preserve the type.

        Show
        alex.behm Alexander Behm added a comment - The simplest solution is to avoid constant folding "CAST(NULL AS TYPE)" expressions specifically. The problem with those is that with our current approach it's very difficult to preserve the type.
        Hide
        alex.behm Alexander Behm added a comment -

        Anuj, do you want to take a stab at this?

        Show
        alex.behm Alexander Behm added a comment - Anuj, do you want to take a stab at this?
        Hide
        grahn Greg Rahn added a comment - - edited

        Even simpler test case:

        create table t as (select cast(null as int) c);
        
        ERROR: AnalysisException: Unsupported type 'null_type' in column 'c' of table 't'
        CAUSED BY: TableLoadingException: Unsupported type 'null_type' in column 'c' of table 't'
        
        Show
        grahn Greg Rahn added a comment - - edited Even simpler test case: create table t as (select cast ( null as int ) c); ERROR: AnalysisException: Unsupported type 'null_type' in column 'c' of table 't' CAUSED BY: TableLoadingException: Unsupported type 'null_type' in column 'c' of table 't'
        Hide
        anujphadke Anuj Phadke added a comment -

        Alexander Behm - Yes I can take this up.

        Show
        anujphadke Anuj Phadke added a comment - Alexander Behm - Yes I can take this up.
        Show
        anujphadke Anuj Phadke added a comment - https://github.com/apache/incubator-impala/commit/8660c404c93673559c639cf7306c0a6f7ab9001f

          People

          • Assignee:
            anujphadke Anuj Phadke
            Reporter:
            bgitline Boris Gitline
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development