[IGNITE-20971] The Ignite process huge memory overhead for tables creation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-beta2
• Fix Version/s: 3.0.0-beta2
• Component/s: general
• Labels: ignite-3
• Assignee: Ivan Bessonov
• Reporter: Igor
• Reviewer: Roman Puchkovskiy
• Created: 27/Nov/23 15:22
• Updated: 08/Dec/23 09:56
• Resolved: 07/Dec/23 15:00

Description

Steps to reproduce

1. Start process with `-Xms4096m -Xmx4096m`
2. Create tables with 5 columns one by one up to 1000.

Expected:
1000 tables are created.

 

Actual:

After ~219 the process was killed by OOM killer due to process took 64GB of memory of available 65GB.

 

Additional information:

OOM killer output:

[Tue Nov 28 18:35:37 2023] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-10002.slice/session-201.scope,task=java,pid=11748,uid=10002
[Tue Nov 28 18:35:37 2023] Out of memory: Killed process 11748 (java) total-vm:97160836kB, anon-rss:64192728kB, file-rss:0kB, shmem-rss:0kB, UID:10002 pgtables:127948kB oom_score_adj:0
[Tue Nov 28 18:35:39 2023] oom_reaper: reaped process 11748 (java), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

pubagent:~$ grep MemTotal /proc/meminfo
MemTotal:       65038556 kB
pubagent:~$ free               
total        used        free      shared  buff/cache   available
Mem:        65038556      321848    64557116         900      159592    64174540
Swap:              0           0           0

GC log
gc.20231128_132812_386649.log

Issue Links

• links to: GitHub Pull Request #2929