[DRILL-2851] Memory LEAK - FLATTEN function fails when input array has 99,999 integer type elements

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 1.2.0
• Component/s: Execution - Flow

Environment

64e3ec52b93e9331aa5179e040eca19afece8317 | DRILL-2611: value vectors should report valid value count | 16.04.2015 @ 13:53:34 EDT

• Assignee: Hanifi Gunes
• Reporter: Khurram Faraaz
• Created: 23/Apr/15 00:51
• Updated: 02/Sep/15 00:11
• Resolved: 06/Jul/15 23:29

Description

FLATTEN function does not return results when input array has 99,999 elements of type integer. Test was run on 4 node cluster on CentOS.

Looks like there is a memory leak somewhere because I see this message in the stack trace. "Failure while closing accountor".

0: jdbc:drill:> select name,flatten(num_list) from `Jsn_Arry100.json`;
Query failed: SYSTEM ERROR: initialCapacity: -2147483648 (expectd: 0+)

[2b1960d3-c9e5-43cc-926e-783257cc1a0a on centos-04.qa.lab:31010]

Error: exception while executing query: Failure while executing query. (state=,code=0)

There are 99,999 integer type elements in the input array.

0: jdbc:drill:> select repeated_count(tmp.num_list) from `Jsn_Arry100.json` tmp;
+------------+
|   EXPR$0   |
+------------+
| 99999      |
+------------+
1 row selected (0.206 seconds)

The below query does NOT return all the 99,999 array elements. Instead query returns incomplete/incorrect results.

0: jdbc:drill:> select tmp.num_list from `Jsn_Arry100.json` tmp; 
+------------+
|  num_list  |
+------------+
| [13729690148580628,33968383451250544,19729687836805948,10887002260554076,59060271975938184,87704035211118608,73861468909987168,88448989981158368,57022772257593232,39886649400114208,59634364188801728,49220606544154264,14668098707176940,1389807151115602,68917737889186128,63305048453386056,65444797852007920,45819687647405048,2572319061962803,30445371906667328,409277773257482,63616924473446912,71267711965641544,39123983528649880,33710976622262344,77269001875184320,21285915577966972,79158399148342640,59927799708140928,86792628837467632,42093378633089096,90092572909620640,27822481748467540,72874902594517600,4613424573716378,30348741516686960,15384400403234324,11312915166709258,67842306780001720,63536034928852224,34064758786460920,26742651581265676,69283348697630136,21337762946874492,33408483778102284,48525199800724472,17366398171254334,78156187420036768,55683717108215368,18931739169089156,53749386072208016,75716769953459472,70124914126143752,59670587242776496,51687272393733984,56590991314575,27798845713791992,57060186084971832,80286905552877744,11576390595076536,67263019646709888,62231475148843856,38916556483991208,56870539861336200,13820727892494552,43440054512663296,43294405699266032,22942812764355256,36368231331952648,52243742256792032,48655336740833488,87161415891865104,74521440214901872,23963586190891468,86994250559679456,64235294682694968,89237837009514944,27981168913939540,86292939130238320,9803308891945780,67756129441807768,47956437308413040,80258260743958512,33828778916469536,8419255582699858,74560162108011744,38543450292892984,21415391273461424,31658676421021728,10459153047723704,73185429768682384,2722949752940800,47317026756664832,54654463675350704,26408396249642988,43938679869946224,56284724555406552,71685791015933536,46001126444228888,68794495595243400,4132552956335268,34688634621761076,35712207345268316,60213985727177096,5309212305048668,23619206841198624,64825327846779600,57007947199911488,47136678928834984,9122467237416458,40980849927903520,20914596281235304,77253379050572112,80560013812121728,46022174419866912,22101913922421504,4767879560244378,7295985537104415,33463848115973120,21159496087635528,7209644495944438,8339341099391089,56466941472321008,49199375007197008,19176844581370920,87713551163634272,68410140933429768,22463584602000640,31007049671139340,88991676818494464,86018450036815872,68296039329719152,40855740519055456,49077682665580104,8114179710795960,19904226756293724,38777905573998560,87116624775839824,64713191586087936,23526229712701268,62999544446558568,61842195984909040,20542838487485340,64036764083509024,1241102073337856,2271939416579819,40485492208390352,32715355572929904,72573056371850208,79275623956295168,63005955205671488,80329641439025648,50636113571403528,84640203909310624,58152135741332672,76874210847790576,45638822712398088,14793955144036874,87502976063529168,58807684142910336,8542862544501658,42269245005614528,58763187554800520,751011850228275,48740246868357488,50972520565647320,72594124941772656,31070553596449896,45234757963903568,59324150925583104,84153005711977232,90541040496368064,71711996735859880,41767774512805840,6237941704680079,10085371064613816,78263530428653888,25649178272406712,78386911979125408,74871831010694784,2501120501320468,59826807125850064,11455550309471406,2789047787534469,90319173073564784,35147753006638328,90980208101318128,9910622567479808,29022215390318684,68516938237020496,13915913719658844,43980253031382328,41811554449342792,30871530529058796,37690170762904656,81110630593395616,14324825121595106,2009494929453865,37214432592899504,17099423225928572,56050443116580560,57187738511658096,30731274919325308,43715257438566712,66086595725233376,72139658658973280,81629806228098832,88214343607466688,11459521311029208,23429003870149408,59435025387927032,87438952800109392,14891802892848088,9676533574567302,27186823028758056,54804348760028984,54924273457109648,89467987292396752,69692650285458240,13851588882257624,21084991327837092,34184135079206584,81287720544047152,68354424116184832,9763366960601098,5076272451255951,60975963817311856,60203372513789336,84076654183130992,23498858210490492,19796619337627596,19703049436223560,10040949502371082,14168760473553624,29560677088141436,7238368756723210,59189735753188576,87403992458731312,12816830478611200,52541905950480408,49643679210487144,74203685134169504,46279848175852064,18972760835253096,83663642472309072,54295553706201856,64608871302467680,41794803336859056,91273876479904320,19510503273546496,21174096823508532,78025664575445136,12118597218456852,13895570027535872,40328409583119688,90090258465666512,47772926087456904,69971593804424344,30961201123684772,24798639486111008,19796544325250448,76841274718788592,83627192334576560,5934726517517281,25204549908243796,71846978502928888,43604846490509336,34110869694768200,47894781247793584,84440194478300448,61511851011957576,61521128641831048,75935420834173200,47754021630636344,21180666053858644,9934584431549716,36958889494931568,54633380674977384,15628243593734082,51392498919665072,14418520607145442,28294203469899224,9231039884803676,70200018940576312,55060314668028264,41805592567813336,69667784789645048,19874163368870584,63457847313701488,33517138148037192,38729100611607000,62264887692937728,75924975833373344,42143671487680760,30051803099963688,35970052888440740,20804318872678156,64202259133559320,89224969496660160,13599192984324056,37593036811603088,89606814956181552,28552227597987452,70488756671457448,17145464048060272,31952855373667268,16719021259572920,65571136190971688,86870559642719104,35455462197754808,70035629262079624,5814237233877566,3667688279824302,32527804673906708,39604742684966176,6653338960970537,18267942973956444,13951321415529974,37616807664818416,245164139914280,37267135279762288,80412924840730896,83320537505825424,28675761855322012,56329261929462256,52347739560820728,84637891335668448,10999384069762058,61447706697887960,40486496111949096,5897125033905091,26138393724872868,26403230650590392,14686912738831780,85000181993850224,32664538987800208,85344655404034752,752110474357944,54964104688025584,85719625463312240,13287910832305798,62660648869455872,74030353151886816,45096117411394600,41444747098005168,53758025544295208,11676918578616546,61776892476600800,34425840976378040,47979985632276600,65220402534819536,47214876583685512,4785522866167747,4996411509661819,82532490890710176,15310371979266592,13487958278328944,72572615495171824,88124137443202448,62974105825355816,22938443003344700,22218760252807556,71463215215220960,24355595948161444,55669834837735688,69044559395815960,22077102505483868,63153013299984136,54350496656156600,90828816267210656,54087476672311104,79873040446604528,52268937199458120,33414164925645640,14624230559059508,63526694004890728,66707816731801704,12985961723467294,51585323788992808,88736963570171344,18097269319441584,2938889277551309,32071836612136672,4874915565971272,27979480399327724,2328631417539911,2019400820084408,1636943567211632,16893692144490528,34307996692585932,3137453755279063,90240966405885680,5190519387913114,91971558378938208,38646106504775936,14100060496946104,35712181109832356,8691252113755187,5389022101506488,39093643829212880,57729646856263080,66298241057795840,48180451448026328,16475799417943920,3528180864814223,62462064545377680,83717871324475840,45190865064772640,46496670780493680,20556297818783632,65867465283654968,36231285292019024,78425984230913872,59310088557323592,19263520242582948,26657084161716696,26768514687177576,34845391769043824,67109862516086352,84787433767086608,32811725219738124,70946466987128088,17487431799052360,29064138224780444,38448830352246448,37919824292475320,65396741238698568,68062317142957816,25215654618159820,77682927719614336,50451906148874064,71198027460152208,81928407285746880,47977106158476096,54798795218991280,87104592426022320,80692836395850320,44742025618412464,83828703034820320,61322244404364440,65852052110978688,46201167582148704,63147684887913728,81947148443265904,33234882570247280,61824203084365664,65677773616676616,34507751060305028,85160735088238096,20250909199245496,83776390609328400,71832337939504784,9156127359953428,73739306112100528,20463550289714472,8097902663668664,57048612187389032,81572091753206912,3829847328411453,50956413862423992,26439809155293092,31034047458962320,76933834719482288,74198948210922976,44622397137614528,39881358494075816,78670828431098720,88941405691527184,72729787251564176,87477963231913024,79711111648840752,12717912830791414,29389029458694524,14376186261304638,50303495048847480,51793414994329632,31332682299419720,6806056236834048,80159745841517264,78893848103037248,13978732633703168,12524246333502792,82480072119463584,7795953954604022,33519837050152828,22501603086029968,73345532642041008,55030778779197,86737038779777952,1203114673054064,69921343463104184,238935739214981,65361843440835200,26231581589378416,16956461555850598,27015530661651252,81284469501562192,75026097714234080,59356950361550776,64852611047367088,74793804210862064,68132958833577168,4452723653819443,41666663396639160,14716222059915520,72348427991948688,59632368262572440,30537619221699932,50647738352100504,21589707182635632,87782956274734896,20618088160030772,80424637973969168,57793644568143288,91595572924028176,12319000882705684,28628328859456872,14914183162316646,55349099487758400,37312488485553504,64143239353460944,9263483809230060,20957503537485436,5003071205474325,62867518655820440,2094345702430361,56644818647364904,35442094691948984,42842292748902200,38576949302537312,86049988424730592,4332711517459917,33778101497672008,69700296469066296,53510614336070992,63483158994184184,63672532023341648,13637502915063480,8151430968818535,75170922637346336,64379628654616872,71343912677494152,70294272306149560,23194619010788016,61075095973093848,64829369100985048,43640706676835688,47391908058263080,57412591849149440,45596372988325296,52315161431461200,49453251968431184,91116412406885840,38139609592712080,59473127849679616,45434093125010000,48405275650237240,69699493929148192,39821008065583872,421570 |
+------------+
1 row selected (0.264 seconds)

Stack trace from drillbit.log

2015-04-23 00:42:29,110 [2ac7c010-3c5c-aab9-c425-a4efcc811fb2:frag:0:0] ERROR o.a.drill.exec.ops.FragmentContext - Fragment Context received failure -- Fragment: 0:0
org.apache.drill.common.exceptions.DrillUserException: SYSTEM ERROR: initialCapacity: -2147483648 (expectd: 0+)

[3818da78-93f5-430a-a8e8-990d819c15bf on centos-04.qa.lab:31010]

        at org.apache.drill.common.exceptions.DrillUserException$Builder.build(DrillUserException.java:115) ~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.common.exceptions.ErrorHelper.wrap(ErrorHelper.java:39) ~[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.ops.FragmentContext.fail(FragmentContext.java:151) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:182) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: java.lang.IllegalArgumentException: initialCapacity: -2147483648 (expectd: 0+)
        at io.netty.buffer.PooledByteBufAllocatorL.validate(PooledByteBufAllocatorL.java:77) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
        at io.netty.buffer.PooledByteBufAllocatorL.directBuffer(PooledByteBufAllocatorL.java:65) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
        at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:227) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:234) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:171) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:358) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.vector.RepeatedBigIntVector$Mutator.addSafe(RepeatedBigIntVector.java:462) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.vector.RepeatedBigIntVector$Mutator.setSafe(RepeatedBigIntVector.java:456) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.test.generated.FlattenerGen207.doEval(FlattenTemplate.java:27) ~[na:na]
        at org.apache.drill.exec.test.generated.FlattenerGen207.flattenRecords(FlattenTemplate.java:94) ~[na:na]
        at org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.doWork(FlattenRecordBatch.java:156) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.flatten.FlattenRecordBatch.innerNext(FlattenRecordBatch.java:122) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:74) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:76) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:64) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:164) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        ... 4 common frames omitted
2015-04-23 00:42:29,112 [2ac7c010-3c5c-aab9-c425-a4efcc811fb2:frag:0:0] INFO  o.a.drill.exec.work.foreman.Foreman - State change requested.  RUNNING --> FAILED
org.apache.drill.common.exceptions.DrillRemoteException: SYSTEM ERROR: initialCapacity: -2147483648 (expectd: 0+)

[36479914-f336-4cb4-992f-5b3aae398c73 on centos-04.qa.lab:31010]

        at org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:235) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:183) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]

...

2015-04-23 00:42:29,136 [2ac7c010-3c5c-aab9-c425-a4efcc811fb2:frag:0:0] WARN  o.a.d.e.w.fragment.FragmentExecutor - Failure while closing out resources
java.lang.IllegalStateException: Failure while closing accountor.  Expected private and shared pools to be set to initial values.  However, one or more were not.  Stats are
        zone    init    allocated       delta
        private 1000000 1000000 0
        shared  9999000000      12146483648     -2147483648.
        at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:301) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.ops.OperatorContext.close(OperatorContext.java:118) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractRecordBatch.cleanup(AbstractRecordBatch.java:170) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.cleanup(AbstractSingleRecordBatch.java:120) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.cleanup(IteratorValidatorBatchIterator.java:148) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.record.AbstractSingleRecordBatch.cleanup(AbstractSingleRecordBatch.java:121) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.cleanup(IteratorValidatorBatchIterator.java:148) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.internalStop(ScreenCreator.java:131) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.stop(ScreenCreator.java:138) ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:209) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:188) [drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]

Issue Links

• is related to: DRILL-3323 Flatten planning rule creates unneeded copy of the list being flattened, causes executuion/allocation issues with large lists
• is related to: DRILL-1944 JsonReader fails to read arrays of size 2000
• links to: ReviewBoard