[KYLIN-2154] source table rows will be skipped if TOPN's group column contains NULL values

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.6.0
• Assignee: Hongbin Ma
• Reporter: Hongbin Ma
• Created: 02/Nov/16 14:26
• Updated: 26/Nov/16 08:12
• Resolved: 04/Nov/16 02:21

Description

topn: group by country and order by sum(price)

if the country column contains NULL values:

2016-11-02 20:23:22,478 ERROR [main] org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase: Insane record: [20111209, \N, 1000122, 101, 9
java.lang.NullPointerException
at org.apache.kylin.common.util.Bytes.toBytes(Bytes.java:351)
at org.apache.kylin.measure.topn.TopNMeasureType$1.valueOf(TopNMeasureType.java:145)
at org.apache.kylin.measure.topn.TopNMeasureType$1.valueOf(TopNMeasureType.java:116)
at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValueOf(BaseCuboidMapperBase.java:186)
at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildValue(BaseCuboidMapperBase.java:157)
at org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMapperBase.java:203)
at org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboidMapper.java:52)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)