[YARN-9717] Add more logging to container-executor about issues with directory creation or permissions

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Peter Bacsko
• Reporter: Szilard Nemeth
• Created: 01/Aug/19 12:16
• Updated: 12/Dec/19 20:57
• Resolved: 23/Sep/19 14:03

Description

During some downstream testing we bumped into some problems with the container executor where an extra logging would be quite helpful when local files and directories could not be created (container-executor.c:1810).

The most important log line could be the following:
There's a function called create_container_directories in container-executor.c.
We should place a log line like this:

Before we're calling:

We have:

if (mkdirs(container_dir, perms) == 0) {
      result = 0;
}

We could add an else statement and add the following log, if creating the directory was not successful:

fprintf(LOGFILE, "Failed to create directory: %s, user: %s", container_dir, user);

This way, CE at least prints the directory itself if we have any permission issue while trying to create a subdirectory or file under it.
If we want to be very precise, some logging into the mkdirs function could also be added as well.

Issue Links

• relates to: YARN-9667 Container-executor.c duplicates messages to stdout