Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16882

NPE when reading hive table with RegexSerde

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Component/s: Connectors / Hive
    • Labels:
      None
    • Environment:

      flink-1.10.0、hive-1.2.0

       

      Description

      When I use sql-client to read hive table with serde 'org.apache.hadoop.hive.serde2.RegexSerDe', it throws a NullPointerException:

      org.apache.flink.connectors.hive.FlinkHiveException: Error happens when deserialize from storage file.
       at org.apache.flink.connectors.hive.read.HiveMapredSplitReader.<init>(HiveMapredSplitReader.java:131)
       at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:121)
       at org.apache.flink.connectors.hive.read.HiveTableInputFormat.open(HiveTableInputFormat.java:56)
       at org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction.run(InputFormatSourceFunction.java:85)
       at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:100)
       at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:63)
       at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:196)
      Caused by: java.lang.NullPointerException
       at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
       at com.google.common.base.Splitter.split(Splitter.java:383)
       at org.apache.hadoop.hive.serde2.RegexSerDe.initialize(RegexSerDe.java:151)
       at org.apache.hadoop.hive.serde2.AbstractSerDe.initialize(AbstractSerDe.java:53)
       at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:521)
       at org.apache.flink.connectors.hive.read.HiveMapredSplitReader.<init>(HiveMapredSplitReader.java:127)
       ... 6 more
      

       

      hive table creation syntax:

      CREATE TABLE test_serde( 
         c0 string COMMENT 'COMMENT-c0', 
         c1 string COMMENT 'COMMENT-c1', 
         c2 string COMMENT 'COMMENT-c2') 
       ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe' 
       WITH SERDEPROPERTIES ( 
        'input.regex' = '([^ ]*) ([^ ]*) ([^ ]*)', 
        'output.format.string' = '%1$s %2$s %3$s'
       )
      

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                leen linweijiang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: