[IGNITE-4037] High memory consumption when executing TeraSort Hadoop example

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.6
• Fix Version/s: 2.0
• Assignee: Vladimir Ozerov
• Reporter: Ivan Veselovsky
• Created: 05/Oct/16 17:26
• Updated: 22/Dec/16 08:30
• Resolved: 22/Dec/16 08:30

Description

When executing TeraSort Hadoop example, we observe high memory consumption that frequently leads to cluster malfunction.

The problem can be reproduced in unit test, even with 1 node, and with not huge input data set as 100Mb.

Dump analysis shows that memory is taken in various queues:
org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService#queue
and
task queue of org.apache.ignite.internal.processors.hadoop.jobtracker.HadoopJobTracker#evtProcSvc .

Since objects stored in these queues hold byte arrays of significant size, memory if consumed very fast.
It looks like real cause of the problem is that some tasks are blocked.

Sub-Tasks

• Spilled map-reduce: map side.: Closed
• Spilled map-reduce: reduce side.: Closed