[HIVE-18398] WITH SERDEPROPERTIES option is broken without org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.2.1
• Component/s: Hive
• Assignee: Unassigned
• Reporter: Rajkumar Singh
• Created: 08/Jan/18 23:23
• Updated: 22/Apr/18 21:40

Description

Steps to reproduce:
1. Create table

create table test_serde(id int,value string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' ESCAPED BY '\\' 

2. show create table produce following output

CREATE TABLE `test_serde`(
  `id` int, 
  `value` string)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY '|' 
WITH SERDEPROPERTIES ( 
  'escape.delim'='\\') 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://hdp262a.hdp.local:8020/apps/hive/warehouse/test_serde'
TBLPROPERTIES (
  'COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\"}', 
  'numFiles'='0', 
  'numRows'='0', 
  'rawDataSize'='0', 
  'totalSize'='0', 
  'transient_lastDdlTime'='1515448894')

3. once you run the create table using the output of show create it ran into the parsing error

NoViableAltException(296@[1876:103: ( tableRowFormatMapKeysIdentifier )?])
	at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
	at org.antlr.runtime.DFA.predict(DFA.java:116)
	.....
FAILED: ParseException line 6:0 cannot recognize input near 'WITH' 'SERDEPROPERTIES' '(' in serde properties specification

4. table create with LazySimpleSerde don't have any such issue.

hive> CREATE TABLE `foo`( 
    > `col` string) 
    > ROW FORMAT SERDE 
    > 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe' 
    > WITH SERDEPROPERTIES ( 
    > 'serialization.encoding'='UTF-8') 
    > STORED AS INPUTFORMAT 
    > 'org.apache.hadoop.mapred.TextInputFormat' 
    > OUTPUTFORMAT 
    > 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' ;
OK
Time taken: 0.375 seconds