Hive
  1. Hive
  2. HIVE-2615

CTAS with literal NULL creates VOID type

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.6.0, 0.7.0, 0.8.0, 0.9.0, 0.10.0, 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Create the table with a column that always contains NULL:

      hive> create table bad as select 1 x, null z from dual;

      Because there's no type, Hive gives it the VOID type:

      hive> describe bad;
      OK
      x int
      z void

      This seems weird, because AFAIK, there is no normal way to create a column of type VOID. The problem is that the table can't be queried:

      hive> select * from bad;
      OK
      Failed with exception java.io.IOException:java.lang.RuntimeException: Internal error: no LazyObject for VOID

      Worse, even if you don't select that field, the query fails at runtime:

      hive> select x from bad;
      ...
      FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

      1. HIVE-2615.1.patch
        2 kB
        Zhuoluo (Clark) Yang

        Issue Links

          Activity

          David Phillips created issue -
          Hide
          David Phillips added a comment -

          I see this as three different issues:

          1. The NULL literal produces a VOID type.
          2. Tables with a VOID type cannot be queried at all.
          3. Queries for tables with a VOID type fail at execution time rather than compile time.

          Some potential solutions:

          1. Treat a VOID type as always being NULL. This is a non-backwards-compatible object inspector change that require changes all over the place, including third party code.
          2. Pick a type for a NULL literal in CTAS. There is no obvious choice for the type, but STRING seems like the best option.
          3. Have the CTAS error out at compile time with a hint for the user to use CAST to specify the type.
          Show
          David Phillips added a comment - I see this as three different issues: The NULL literal produces a VOID type. Tables with a VOID type cannot be queried at all. Queries for tables with a VOID type fail at execution time rather than compile time. Some potential solutions: Treat a VOID type as always being NULL. This is a non-backwards-compatible object inspector change that require changes all over the place, including third party code. Pick a type for a NULL literal in CTAS. There is no obvious choice for the type, but STRING seems like the best option. Have the CTAS error out at compile time with a hint for the user to use CAST to specify the type.
          Hide
          Zhuoluo (Clark) Yang added a comment -

          I think option 3 is the better choice, from David Phillips says.
          Is any body working on this issue?

          Show
          Zhuoluo (Clark) Yang added a comment - I think option 3 is the better choice, from David Phillips says. Is any body working on this issue?
          Zhuoluo (Clark) Yang made changes -
          Field Original Value New Value
          Assignee Zhuoluo (Clark) Yang [ clarkyzl ]
          Zhuoluo (Clark) Yang made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Zhuoluo (Clark) Yang made changes -
          Affects Version/s 0.6.0 [ 12314524 ]
          Zhuoluo (Clark) Yang made changes -
          Component/s Query Processor [ 12312586 ]
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Attach a patch.
          The checks after result schema is generated.
          if CTAS and contains void, it raise an exception and ask user to cast the type.

          Show
          Zhuoluo (Clark) Yang added a comment - Attach a patch. The checks after result schema is generated. if CTAS and contains void, it raise an exception and ask user to cast the type.
          Zhuoluo (Clark) Yang made changes -
          Attachment HIVE-2615.1.patch [ 12586062 ]
          Show
          Zhuoluo (Clark) Yang added a comment - https://reviews.apache.org/r/11622/
          Hide
          Zhuoluo (Clark) Yang added a comment -

          Would any committer review this issue?

          Show
          Zhuoluo (Clark) Yang added a comment - Would any committer review this issue?
          Zhuoluo (Clark) Yang made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Hide
          Ashutosh Chauhan added a comment -

          Instead of in SemanticAnalyzer, better place to do this check is in TypeCheckProcFactory.java

          Show
          Ashutosh Chauhan added a comment - Instead of in SemanticAnalyzer, better place to do this check is in TypeCheckProcFactory.java
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Affects Version/s 0.11.0 [ 12323587 ]
          Affects Version/s 0.10.0 [ 12320745 ]
          Affects Version/s 0.9.0 [ 12317742 ]
          Affects Version/s 0.8.0 [ 12316178 ]
          Affects Version/s 0.7.0 [ 12315150 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Hide
          Johndee Burks added a comment -

          An example of what the cast would look like:

          create table <new_table> as select <column>, cast(null as <type>) <column_name> from <table_name>;

          create table null_test as select user, cast(null as bigint) test from a;

          Show
          Johndee Burks added a comment - An example of what the cast would look like: create table <new_table> as select <column>, cast(null as <type>) <column_name> from <table_name>; create table null_test as select user, cast(null as bigint) test from a;
          Hide
          Harsh J added a comment -

          This has been fixed via HIVE-4172. Resolving as duplicate.

          Show
          Harsh J added a comment - This has been fixed via HIVE-4172 . Resolving as duplicate.
          Harsh J made changes -
          Link This issue duplicates HIVE-4172 [ HIVE-4172 ]
          Harsh J made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 0.12.0 [ 12324312 ]
          Resolution Duplicate [ 3 ]

            People

            • Assignee:
              Zhuoluo (Clark) Yang
              Reporter:
              David Phillips
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development