[IMPALA-935] NPE when planning a query

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 1.3
• Fix Version/s: Impala 1.3.1
• Labels: NPE impala planner
• Environment: CDH5
• Assignee: Alexander Behm
• Reporter: Iván de Prado
• Created: 04/Apr/14 12:29
• Updated: 09/Apr/14 18:48
• Resolved: 07/Apr/14 23:40

Description

After upgrading from CDH5-beta2 to CDH5 final release, some queries in Impala started to failed.

Particularly, the query:

INSERT OVERWRITE TABLE a_day_idadvertiser
PARTITION(ddate)
SELECT

  coalesce(g.idadvertiser, c.idadvertiser, i.idadvertiser),
  coalesce(max(i.hits), 0),
  coalesce(max(c.clks), 0),
  coalesce(max(g.goals), 0),
  coalesce(max(g.val), 0) + coalesce(max(c.val), 0) + coalesce(max(i.val), 0),
  coalesce(g.ddate, c.ddate, i.ddate)

FROM

  (SELECT ddate, idadvertiser, count(*) goals, sum(value) val
   FROM goals
   WHERE ddate >= '2014-04-03' AND ddate <= '2014-04-05'
   GROUP BY ddate, idadvertiser) g

  FULL OUTER JOIN

  (SELECT ddate, idadvertiser, count(*) clks, sum(value) val
   FROM clicks
   WHERE click_status = 1 AND ddate >= '2014-04-03' AND ddate <= '2014-04-05'
   GROUP BY ddate, idadvertiser) c

  ON (g.ddate = c.ddate AND g.idadvertiser = c.idadvertiser)
  FULL OUTER JOIN

  (SELECT ddate, idadvertiser, count(*) hits, sum(value) val
   FROM impressions
   WHERE ddate >= '2014-04-03' AND ddate <= '2014-04-05'
   GROUP BY ddate, idadvertiser) i

  ON (i.ddate = c.ddate AND i.idadvertiser = c.idadvertiser)

GROUP BY COALESCE(g.ddate, c.ddate, i.ddate), coalesce(g.idadvertiser, c.idadvertiser, i.idadvertiser);  

Fails, with the following error in logs:

java.lang.NullPointerException
	at com.cloudera.impala.planner.PlanFragment.getDestFragment(PlanFragment.java:253)
	at com.cloudera.impala.planner.Planner.createHashJoinFragment(Planner.java:521)
	at com.cloudera.impala.planner.Planner.createPlanFragments(Planner.java:247)
	at com.cloudera.impala.planner.Planner.createPlanFragments(Planner.java:236)
	at com.cloudera.impala.planner.Planner.createPlanFragments(Planner.java:236)
	at com.cloudera.impala.planner.Planner.createPlanFragments(Planner.java:145)
	at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:595)
	at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:121)

If even fails when doing an explain. So the bug seems to be at the Planner. Any workaround for that?

Impala version: (Shell build version: Impala Shell vcdh5-1.3.0 (40e1b62) built on Tue Mar 25 13:46:44 PDT 2014)