Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10161

TestRouterWebServicesREST is corrupting STDOUT

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.10.0, 3.2.1
    • Fix Version/s: 3.0.4, 3.3.0, 3.2.2, 3.1.4, 2.10.1
    • Component/s: yarn
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestRouterWebServicesREST is creating processes that inherit stdin/stdout from the current process, so the output from those jobs goes into the standard output of mvn test.

      Here's an example from a recent build:

      [WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file /testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/target/surefire-reports/2020-02-24T08-00-54_776-jvmRun1.dumpstream
      [INFO] Tests run: 41, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.644 s - in org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST
      [WARNING] ForkStarter IOException: 506 INFO  [main] resourcemanager.ResourceManager (LogAdapter.java:info(49)) - STARTUP_MSG: 
      522 INFO  [main] resourcemanager.ResourceManager (LogAdapter.java:info(49)) - registered UNIX signal handlers for [TERM, HUP, INT]
      876 INFO  [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2588)) - core-site.xml not found
      879 INFO  [main] security.Groups (Groups.java:refresh(402)) - clearing userToGroupsMap cache
      930 INFO  [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2588)) - resource-types.xml not found
      930 INFO  [main] resource.ResourceUtils (ResourceUtils.java:addResourcesFileToConf(421)) - Unable to find 'resource-types.xml'.
      940 INFO  [main] resource.ResourceUtils (ResourceUtils.java:addMandatoryResources(126)) - Adding resource type - name = memory-mb, units = Mi, type = COUNTABLE
      940 INFO  [main] resource.ResourceUtils (ResourceUtils.java:addMandatoryResources(135)) - Adding resource type - name = vcores, units = , type = COUNTABLE
      974 INFO  [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2591)) - found resource yarn-site.xml at file:/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/target/test-classes/yarn-site.xml
      001 INFO  [main] event.AsyncDispatcher (AsyncDispatcher.java:register(227)) - Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
      053 INFO  [main] security.NMTokenSecretManagerInRM (NMTokenSecretManagerInRM.java:<init>(75)) - NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
      060 INFO  [main] security.RMContainerTokenSecretManager (RMContainerTokenSecretManager.java:<init>(79)) - ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
      ... 

      It seems like these processes should be rerouting stdout/stderr to a file instead of dumping it to the console.

        Attachments

        1. YARN-10161.001.patch
          4 kB
          Jim Brennan
        2. YARN-10161.002.patch
          4 kB
          Jim Brennan
        3. YARN-10161.003.patch
          4 kB
          Jim Brennan

          Activity

            People

            • Assignee:
              Jim_Brennan Jim Brennan
              Reporter:
              Jim_Brennan Jim Brennan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: