Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24621

TEXT and varchar datatype does not support unicode encoding in MSSQL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 4.0.0
    • Fix Version/s: None
    • Component/s: Standalone Metastore
    • Labels:
      None

      Description

      Why Unicode is required?
      In following example the Chinese character cannot be properly interpreted. 

      CREATE VIEW `test_view` AS select `test_tbl_char`.`col1` from `test_db5`.`test_tbl_char` where `test_tbl_char`.`col1`='你好'; 
      
      show create table test_view;
      +----------------------------------------------------+
      |                   createtab_stmt                   |
      +----------------------------------------------------+
      | CREATE VIEW `test_view` AS select `test_tbl_char`.`col1` from `test_db5`.`test_tbl_char` where `test_tbl_char`.`col1`='??' |
      +----------------------------------------------------+ 

       
      This issue comes because TBLS is defined as follows:
       
      CREATE TABLE TBLS
      (
      TBL_ID bigint NOT NULL,
      CREATE_TIME int NOT NULL,
      DB_ID bigint NULL,
      LAST_ACCESS_TIME int NOT NULL,
      OWNER nvarchar(767) NULL,
      OWNER_TYPE nvarchar(10) NULL,
      RETENTION int NOT NULL,
      SD_ID bigint NULL,
      TBL_NAME nvarchar(256) NULL,
      TBL_TYPE nvarchar(128) NULL,
      VIEW_EXPANDED_TEXT text NULL,
      VIEW_ORIGINAL_TEXT text NULL,
      IS_REWRITE_ENABLED bit NOT NULL DEFAULT 0,
      WRITE_ID bigint NOT NULL DEFAULT 0
      );

      Text data type does not support unicode encoding irrespective of collation
      varchar data type does not support unicode encoding prior to SQL Server 2019. Also UTF8 enabled Collation needs to be defined for use of unicode characters.

        Attachments

          Activity

            People

            • Assignee:
              gupta.nikhil0007 Nikhil Gupta
              Reporter:
              gupta.nikhil0007 Nikhil Gupta
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: