[HIVE-11068] Hive throws OOM in client side

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Prasanth Jayachandran
• Reporter: Rajesh Balamohan
• Created: 22/Jun/15 03:41
• Updated: 10/Jul/15 21:28
• Resolved: 07/Jul/15 22:21

Description

Hive build: (Latest on Jun 21. commit 142426394cfdc8a1fea51f7642c63f43f36b0333).
Query: Query 64 TPC-DS (https://github.com/cartershanklin/hive-testbench/blob/master/sample-queries-tpcds/query64.sql)

Hive throws following OOM in client side.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
  at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149)
  at java.lang.StringCoding.decode(StringCoding.java:193)
  at java.lang.String.<init>(String.java:414)
  at java.lang.String.<init>(String.java:479)
  at org.apache.hadoop.hive.ql.exec.Utilities.serializeExpression(Utilities.java:799)
  at org.apache.hadoop.hive.ql.plan.TableScanDesc.setFilterExpr(TableScanDesc.java:153)
  at org.apache.hadoop.hive.ql.ppd.OpProcFactory.pushFilterToStorageHandler(OpProcFactory.java:901)
  at org.apache.hadoop.hive.ql.ppd.OpProcFactory.createFilter(OpProcFactory.java:818)
  at org.apache.hadoop.hive.ql.ppd.OpProcFactory.createFilter(OpProcFactory.java:788)
  at org.apache.hadoop.hive.ql.ppd.OpProcFactory$TableScanPPD.process(OpProcFactory.java:388)
  at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:133)
  at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
  at org.apache.hadoop.hive.ql.ppd.PredicatePushDown.transform(PredicatePushDown.java:135)
  at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:192)
  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10171)
  at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:207)
  at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
  at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1124)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1172)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1061)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1051)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:311)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:409)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:425)

Issue Links

• is related to: HIVE-10940 HiveInputFormat::pushFilters serializes PPD objects for each getRecordReader call