Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13708

Create table should verify datatypes supported by the serde

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Planning
    • Labels:
      None

      Description

      As Alon Goldshuv mentioned in HIVE-7777.
      Create table with serde such as OpenCSVSerde allows for creation of table with columns of arbitrary types. But 'describe table' would still return string datatypes, and so does selects on the table.
      This is misleading and would result in users not getting intended results.
      The create table ideally should disallow the creation of such tables with unsupported types.

      Example posted by Alon Goldshuv in HIVE-7777 -

      CREATE EXTERNAL TABLE test (totalprice DECIMAL(38,10)) 
      ROW FORMAT SERDE 'com.bizo.hive.serde.csv.CSVSerde' with 
      serdeproperties ("separatorChar" = ",","quoteChar"= "'","escapeChar"= "\\") 
      STORED AS TEXTFILE 
      LOCATION '<some location>' 
      tblproperties ("skip.header.line.count"="1");
      

      Now consider this sql:

      hive> select min(totalprice) from test;

      in this case given my data, the result should have been 874.89, but the actual result became 100001.57 (as it is first according to byte ordering of a string type). this is a wrong result.

      hive> desc extended test;
      OK
      o_totalprice string from deserializer
      ...

        Attachments

        1. HIVE-13708.1.patch
          4 kB
          Hari Sankar Sivarama Subramaniyan
        2. HIVE-13708.2.patch
          9 kB
          Hari Sankar Sivarama Subramaniyan
        3. HIVE-13708.3.patch
          9 kB
          Hari Sankar Sivarama Subramaniyan
        4. HIVE-13708.4.patch
          9 kB
          Hari Sankar Sivarama Subramaniyan

          Issue Links

            Activity

              People

              • Assignee:
                hsubramaniyan Hari Sankar Sivarama Subramaniyan
                Reporter:
                thejas Thejas M Nair
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: