[HADOOP-17438] Increase docker memory limit in Jenkins

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.1, 3.4.0, 2.10.2, 3.2.3
• Component/s: build, scripts, test, yetus
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Ahmed Hussein
• Reporter: Ahmed Hussein
• Created: 17/Dec/20 16:26
• Updated: 10/Jun/21 07:57
• Resolved: 12/Jan/21 06:35

Description

Yetus keeps failing with OOM.

 

unable to create new native thread

java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:717)
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
	at java.util.concurrent.ThreadPoolExecutor.ensurePrestart(ThreadPoolExecutor.java:1603)
	at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:334)
	at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:533)
	at org.apache.maven.surefire.booter.ForkedBooter.launchLastDitchDaemonShutdownThread(ForkedBooter.java:369)
	at org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:333)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:145)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

 

This jira to increase the memory limit from 20g to 22g.

Note: This is only a workaround to get things more productive. If this change reduces the frequency of the OOM failure, there must be a follow-up profile the runtime to figure out which components are causing the docker to run out of memory.

CC: aajisaka, elgoiri, weichiu, ebadger, tasanuma, iwasakims, ayushtkn, inigoiri

Issue Links

• is required by: HADOOP-17224 Install Intel ISA-L library in Dockerfile
• links to: GitHub Pull Request #2560