[HCATALOG-630] org.apache.hcatalog.data.JsonSerDe cannot handle maps with numeric values

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.4, 0.5
• Component/s: storage handlers
• Labels: JsonSerDe
• Assignee: Unassigned
• Reporter: Dmitry Vasilenko
• Created: 13/Mar/13 18:36
• Updated: 01/Oct/15 23:21

Description

Steps to replicate:

1) Create a Hive table

CREATE TABLE test_map (test map<STRING,INT> )
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;

2) Load the data:

LOAD DATA LOCAL INPATH '/home/dvasilen/Downloads/QA_TEST/test_map.json' OVERWRITE INTO TABLE test_map;

3) The test_map.json contains the single line:

{"test":{"key":1}}

4) Run "select * from test_map;" from Hive client

5) Note the following exception being thrown:

13/03/13 13:34:20 ERROR CliDriver: Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
java.io.IOException: org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:466)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1387)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:270)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:755)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:600)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:171)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:506)
... 13 more
Caused by: org.codehaus.jackson.JsonParseException: Current token (FIELD_NAME) not numeric, can not use numeric value accessors
at [Source: java.io.ByteArrayInputStream@2be32be3; line: 1, column: 17]
at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1291)
at org.codehaus.jackson.impl.JsonParserMinimalBase._reportError(JsonParserMinimalBase.java:385)
at org.codehaus.jackson.impl.JsonNumericParserBase._parseNumericValue(JsonNumericParserBase.java:399)
at org.codehaus.jackson.impl.JsonNumericParserBase.getIntValue(JsonNumericParserBase.java:254)
at org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:218)
at org.apache.hcatalog.data.JsonSerDe.extractCurrentField(JsonSerDe.java:279)
at org.apache.hcatalog.data.JsonSerDe.populateRecord(JsonSerDe.java:188)
at org.apache.hcatalog.data.JsonSerDe.deserialize(JsonSerDe.java:167)
... 14 more

If you change the value type to string

CREATE TABLE test_map (test map<STRING,STRING> )
ROW FORMAT SERDE 'org.apache.hcatalog.data.JsonSerDe'
STORED AS TEXTFILE;

and

{"test":{"key":"string"}}

then it works correctly.

I see the same issue in 0.4.0 as well as 0.5.0.

Issue Links

• duplicates: HIVE-12012 select query on json table with map containing numeric values fails