Changes:

container-executor.c:
1. make mkdirs more compatible, remove usage of mkdirat/openat
2. use sysconf() to get LOGIN_NAME_MAX
3. macosx doesn't have fcloseall, so close all opened fds on macosx
4. disable cgroup on macosx

test-container-executor.c:
1. macosx do not have user bin, skip a check
2. change /etc/passwd(not exists on mac) to /bin/ls

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12650514/YARN-2161.v1.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3991//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3991//console

This message is automatically generated.

Looks good for the most part! A few minor things:

+#if defined(__MACH__)
+  // only those fds are opened assuming no bug
+  fclose(LOGFILE);
+  fclose(ERRORFILE);
+  fclose(stdin);
+  fclose(stdout);
+  fclose(stderr);
+#else
   fcloseall();
+#endif

It would probably be better to detect if fcloseall is a valid function. If not, then use the manual fclose. This would allow the code to run on not just OS X but other systems as well.

+#if defined(__FreeBSD__) || defined(__MACH__)
+  fprintf(LOGFILE, "Failed to mount cgroup controller, not supported\n");
+  return -1;
+#else

I sort of prefer the fix from YARN-1327 here, where it does +#ifndef __linux instead.

+#if !defined(__MACH__) // macosx do not have user bin
   if (check_user("bin") == NULL) {
     printf("FAIL: failed check for whitelisted system user bin\n");
     exit(1);
   }
+#endif

Rather than ignore this test completely, can we chose a different user instead?

Also, can you incorporate the other changes from YARN-1327 so that we can do one commit?

Thanks!

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12650514/YARN-2161.v1.patch
against trunk revision b93d960.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5089//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5089//console

This message is automatically generated.

Thanks for the review Allen. Attach new version of the patch, changes:
1. use cmake CHECK_FUNCTION_EXISTS to check fcloseall exists
2. change user bin to user daemon (which both linux and macosx have)
3. add some fix in YARN-1327 (setpgid and libgen.h)

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12670922/YARN-2161.v2.patch
against trunk revision ef784a2.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5094//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5094//console

This message is automatically generated.

+1 lgtm. Committing to branch-2 and trunk.

Thanks!

FAILURE: Integrated in Hadoop-Yarn-trunk #691 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/691/)
YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1882 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1882/)
YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
• hadoop-yarn-project/CHANGES.txt

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1907 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1907/)
YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt

Binglin Chang Allen Wittenauer
For fixing YARN-2701, i need to revert the native code changes for mkdirs in container-executor.c

The changes for mkdirs in container-executor.c bring the race condition when two containers are trying to check and create directory at the same time.

Hi Xuan Gong, sorry for break the code. I see in YARN-2701 you already have fix code, but decide to revert the code in the end to be more safe, but this breaks the mac build, how about use #ifdef to use old code when compiling on glibc > 2.10(http://linux.die.net/man/2/openat) and use your fixing code otherwise?

I've re-opened YARN-2701.