OK I understand now. The original description of the issue by the person who found it was very vague.
The issue is that yarn-$YARN_IDENT_STRING-$command-$HOSTNAME.out is where stdout of the command is placed, and that even though yarn.log.dir, yarn.log.file, and yarn.root.logger system properties are being set they are not being used by the log4j configuration. The log4j configuration defaults to CONSOLE which outputs the logs to stdout which in turn goes to yarn-$YARN_IDENT_STRING-$command-$HOSTNAME.out.
The fix is to either update the log4j conf to use the yarn variants of these variables, or to set the hadoop equivalents of these variables instead. I assume from your comment that we need to switch them over to use the hadoop variants. Is this correct?