[YARN-10495] make the rpath of container-executor configurable

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.3.1
• Component/s: yarn
• Assignee: angerszhu
• Reporter: angerszhu
• Created: 19/Nov/20 09:20
• Updated: 25/Apr/23 15:12
• Resolved: 08/Dec/20 00:54

Description

In  https://issues.apache.org/jira/browse/YARN-9561 we add dependency on crypto to container-executor, we meet a case that in our jenkins machine, we have libcrypto.so.1.0.0  in shared lib env. but in our nodemanager machine we don't have  libcrypto.so.1.0.0  but libcrypto.so.1.1.

We use a  internal custom dynamic link library environment /usr/lib/x86_64-linux-gnu

and we build hadoop with parameter as blow

 -Drequire.openssl -Dbundle.openssl -Dopenssl.lib=/usr/lib/x86_64-linux-gnu

 

Under jenkins machine shared lib library path /usr/lib/x86_64-linux-gun(where is libcrypto)

-rw-r--r-- 1 root root   240136 Nov 28  2014 libcroco-0.6.so.3.0.1
-rw-r--r-- 1 root root    54550 Jun 18  2017 libcrypt.a
-rw-r--r-- 1 root root  4306444 Sep 26  2019 libcrypto.a
lrwxrwxrwx 1 root root       18 Sep 26  2019 libcrypto.so -> libcrypto.so.1.0.0
-rw-r--r-- 1 root root  2070976 Sep 26  2019 libcrypto.so.1.0.0
lrwxrwxrwx 1 root root       35 Jun 18  2017 libcrypt.so -> /lib/x86_64-linux-gnu/libcrypt.so.1
-rw-r--r-- 1 root root      298 Jun 18  2017 libc.so

 

Under nodemanager shared lib library path /usr/lib/x86_64-linux-gun(where is libcrypto)

-rw-r--r--  1 root root    55852 2��   7  2019 libcrypt.a
-rw-r--r--  1 root root  4864244 9��  28  2019 libcrypto.a
lrwxrwxrwx  1 root root       16 9��  28  2019 libcrypto.so -> libcrypto.so.1.1
-rw-r--r--  1 root root  2504576 12�� 24  2019 libcrypto.so.1.0.2
-rw-r--r--  1 root root  2715840 9��  28  2019 libcrypto.so.1.1
lrwxrwxrwx  1 root root       35 2��   7  2019 libcrypt.so -> /lib/x86_64-linux-gnu/libcrypt.so.1
-rw-r--r--  1 root root      298 2��   7  2019 libc.so

 We build container-executor with 

The  libcrypto.so 's version is not same case error when we start nodemanager

 

.. 3 more Caused by: org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationException: ExitCodeException exitCode=127: /home/hadoop/hadoop/bin/container-executor: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:182) at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:208) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:306) ... 4 more Caused by: ExitCodeException exitCode=127: /home/hadoop/hadoop/bin/container-executor: error while loading shared libraries: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008) at org.apache.hadoop.util.Shell.run(Shell.java:901) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213) at org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:154) ... 6 more 

 

We should make RPATH of container-executor configurable to solve this problem 

Issue Links

• relates to: YARN-9561 Add C changes for the new RuncContainerRuntime