Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9350

Change log level to WARN when ShellCommandExecutor.execute() occurs exception

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0, 2.8.5
    • Fix Version/s: None
    • Component/s: yarn
    • Labels:
    • Flags:
      Important

      Description

      When method ShellCommandExecutor.execute() occurs exception , it has inconsistent log level practices in different places as following :

      //hadoop-2.8.5-src\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-common\src\main\java\org\apache\hadoop\yarn\util\WindowsBasedProcessTree.java
      //log statement line number:69
      //log level:error
      try {
         shellExecutor.execute();
      } catch (IOException e) {
         LOG.error(StringUtils.stringifyException(e));
      } finally {
         String output = shellExecutor.getOutput();
         if (output != null &&
         output.contains("Prints to stdout a list of processes in the task")) {
         return true;
      }
      
      //hadoop-2.8.5-src\hadoop-common-project\hadoop-common\src\main\java\org\apache\hadoop\util\NodeHealthScriptRunner.java
      //log statement line number:116
      //log level:warn
      try {
        shexec.execute();
      } catch (ExitCodeException e) {
        ......
      } catch (Exception e) {
         LOG.warn("Caught exception : " + e.getMessage());
         if (!shexec.isTimedOut()) {
           status = HealthCheckerExitStatus.FAILED_WITH_EXCEPTION;
         } else {
           status = HealthCheckerExitStatus.TIMED_OUT;
         }
         exceptionStackTrace = StringUtils.stringifyException(e);
      } finally {
         ......
      }
      
      //hadoop-2.8.5-src\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-server\hadoop-yarn-server-nodemanager\src\main\java\org\apache\hadoop\yarn\server\nodemanager\containermanager\linux\privileged\PrivilegedOperationExecutor.java
      //log statement line number:179
      //log level:warn
      try {
         exec.execute();
         if (LOG.isDebugEnabled()) {
           LOG.debug("command array:");
           LOG.debug(Arrays.toString(fullCommandArray));
           LOG.debug("Privileged Execution Operation Output:");
           LOG.debug(exec.getOutput());
         }
      } catch (ExitCodeException e) {
         ......
      } catch (IOException e) {
      LOG.warn("IOException executing command: ", e);
      throw new PrivilegedOperationException(e);
      }

      There are 2 similar code snippets assign WARN level, when execute() occurs exception, simultaneously only 1 code snippet chooses ERROR level for same situation. Therefore, I think this one log statement is more likely to be assigned WARN level.

        Attachments

        1. NodeHealthScriptRunner.java
          10 kB
          Anuhan Torgonshar
        2. PrivilegedOperationExecutor.java
          10 kB
          Anuhan Torgonshar
        3. WindowsBasedProcessTree.java
          9 kB
          Anuhan Torgonshar

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              OneisAll Anuhan Torgonshar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: