[FLINK-16882] NPE when reading hive table with RegexSerde

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.10.0
• Component/s: Connectors / Hive

Environment

flink-1.10.0、hive-1.2.0

 

• Assignee: Unassigned
• Reporter: linweijiang
• Created: 31/Mar/20 08:54
• Updated: 31/Mar/20 09:37
• Resolved: 31/Mar/20 09:02

Description

When I use sql-client to read hive table with serde 'org.apache.hadoop.hive.serde2.RegexSerDe', it throws a NullPointerException:

org.apache.flink.connectors.hive.FlinkHiveException: Error happens when deserialize from storage file.
 at org.apache.flink.connectors.hive.read.HiveMapredSplitReader.<init>(HiveMapredSplitReader.java:131)
 at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:121)
 at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:56)
 at org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:85)
 at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:100)
 at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:63)
 at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:196)
Caused by: java.lang.NullPointerException
 at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
 at com.google.common.base.Splitter.split(Splitter.java:383)
 at org.apache.hadoop.hive.serde2.RegexSerDe.initialize(RegexSerDe.java:151)
 at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:53)
 at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:521)
 at org.apache.flink.connectors.hive.read.HiveMapredSplitReader.<init>(HiveMapredSplitReader.java:127)
 ... 6 more

 

hive table creation syntax：

CREATE TABLE test_serde( 
   c0 string COMMENT 'COMMENT-c0', 
   c1 string COMMENT 'COMMENT-c1', 
   c2 string COMMENT 'COMMENT-c2') 
 ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe' 
 WITH SERDEPROPERTIES ( 
  'input.regex' = '([^ ]*) ([^ ]*) ([^ ]*)', 
  'output.format.string' = '%1$s %2$s %3$s'
 )

 

Issue Links

• is duplicated by: FLINK-16767 Failed to read Hive table with RegexSerDe