[YARN-8622] NodeManager native build fails due to getgrouplist not found on macOS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0, 3.3.0
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Component/s: nodemanager

Environment

Darwin 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
Apple LLVM version 9.1.0 (clang-902.0.39.2)

• Target Version/s: 3.3.0, 3.2.1, 3.1.3
• Assignee: Siyao Meng
• Reporter: Ewan Higgs
• Created: 03/Aug/18 13:29
• Updated: 02/Oct/19 17:13
• Resolved: 18/Apr/19 23:02

Description

Usage of getgrouplist() is added in YARN-7221 and should affect Hadoop 3.2.0 and later.

Compiler:

$ /Library/Developer/CommandLineTools/usr/bin/c++ --version
Apple LLVM version 9.1.0 (clang-902.0.39.2)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Build line:

[WARNING] /Library/Developer/CommandLineTools/usr/bin/c++   -g -O2 -Wall -pthread -D_FILE_OFFSET_BITS=64 -Wl,-search_paths_first -Wl,-headerpad_max_install_names   CMakeFiles/test-oom-listener.dir/main/native/oom-listener/impl/oom_listener.c.o CMakeFiles/test-oom-listener.dir/main/native/oom-listener/test/oom_listener_test_main.cc.o  -o test/test-oom-listener libgtest.a -lrt 

Error message:

...
[WARNING] /Users/ehiggs/src/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:1264:12: error: no matching function for call to 'getgrouplist'
[WARNING]   int rc = getgrouplist(user, pw->pw_gid, groups, &ngroups);
[WARNING]            ^~~~~~~~~~~~
[WARNING] /usr/include/unistd.h:653:6: note: candidate function not viable: no known conversion from 'gid_t *' (aka 'unsigned int *') to 'int *' for 3rd argument
[WARNING] int      getgrouplist(const char *, int, int *, int *);
[WARNING]          ^
[WARNING] In file included from /Users/ehiggs/src/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc:24:
[WARNING] /Users/ehiggs/src/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:1271:9: error: no matching function for call to 'getgrouplist'
[WARNING]     if (getgrouplist(user, pw->pw_gid, groups, &ngroups) == -1) {
[WARNING]         ^~~~~~~~~~~~
[WARNING] /usr/include/unistd.h:653:6: note: candidate function not viable: no known conversion from 'gid_t *' (aka 'unsigned int *') to 'int *' for 3rd argument
[WARNING] int      getgrouplist(const char *, int, int *, int *);
[WARNING]          ^
[WARNING] 2 warnings and 2 errors generated.
[WARNING] make[2]: *** [CMakeFiles/cetest.dir/main/native/container-executor/test/utils/test_docker_util.cc.o] Error 1
[WARNING] make[1]: *** [CMakeFiles/cetest.dir/all] Error 2
[WARNING] make: *** [all] Error 2

Issue Links

• is caused by: YARN-7221 Add security check for privileged docker container
• relates to: YARN-9487 NodeManager native build shouldn't link against librt on macOS