[DRILL-5857] Fix NumberFormatException in Hive unit tests

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Storage - Hive
• Labels: ready-to-commit
• Assignee: Vova Vysotskyi
• Reporter: Vova Vysotskyi
• Reviewer: Arina Ielchiieva
• Created: 09/Oct/17 10:54
• Updated: 11/Oct/17 18:00
• Resolved: 11/Oct/17 18:00

Description

Though all unit tests pass, it seems that number of hive tests have errors:

11:40:02.558 [262831fd-444d-57ee-b528-8a01a6a1c0a1:foreman] ERROR o.a.d.e.s.hive.HiveMetadataProvider - Failed to parse Hive stats in metastore.
java.lang.NumberFormatException: null
	at java.lang.Long.parseLong(Long.java:404) ~[na:1.7.0_131]
	at java.lang.Long.valueOf(Long.java:540) ~[na:1.7.0_131]
	at org.apache.drill.exec.store.hive.HiveMetadataProvider.getStatsFromProps(HiveMetadataProvider.java:211) [classes/:na]
	at org.apache.drill.exec.store.hive.HiveMetadataProvider.getStats(HiveMetadataProvider.java:100) [classes/:na]
	at org.apache.drill.exec.store.hive.HiveScan.getScanStats(HiveScan.java:229) [classes/:na]
	at org.apache.drill.exec.physical.base.AbstractGroupScan.getScanStats(AbstractGroupScan.java:79) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.logical.DrillScanRel.computeSelfCost(DrillScanRel.java:159) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.calcite.rel.metadata.RelMdPercentageOriginalRows.getNonCumulativeCost(RelMdPercentageOriginalRows.java:162) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost_$(Unknown Source) [janino-2.7.6.jar:na]
	at GeneratedMetadataHandler_NonCumulativeCost.getNonCumulativeCost(Unknown Source) [janino-2.7.6.jar:na]
	at org.apache.calcite.rel.metadata.RelMetadataQuery.getNonCumulativeCost(RelMetadataQuery.java:258) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.getCost(VolcanoPlanner.java:1122) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements0(RelSubset.java:365) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.RelSubset.propagateCostImprovements(RelSubset.java:348) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet(VolcanoPlanner.java:1840) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1772) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:1026) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1046) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1953) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:138) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:213) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.drill.exec.planner.logical.DrillPushProjIntoScan.onMatch(DrillPushProjIntoScan.java:90) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:811) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:310) [calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:401) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:343) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:242) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:292) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:169) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:131) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1050) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:280) [drill-java-exec-1.12.0-SNAPSHOT.jar:1.12.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_131]
	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_131]

This issue appeared after the fix for DRILL-4194, but there were no errors in the console since all logs were written into the file.
After DRILL-5761 these errors appeared in the console since logger appender was changed into STDOUT.

This issue was noticed by arina

Issue Links

• links to: GitHub Pull Request #980