Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3502

Review all Service.stop() operations and make sure that they work before a service is started

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.24.0
    • Fix Version/s: 3.0.0, 2.1.0-beta
    • Component/s: mrv2
    • Labels:
      None

      Description

      MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by

      1. not calling other things if the other things are null
      2. not being re-entrant (i.e. make synchronized if possible),

      Maybe

      1. have a StopService operation that only stops a service if it is live
      2. factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing

      my first iteration will skip the sync though it's something to consider.

      Testing: try to create each instance; call stop() straight after construction.

      1. MAPREDUCE-3502.patch
        45 kB
        Steve Loughran
      2. MAPREDUCE-3502.patch
        17 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          Steve Loughran added a comment -

          First walk through; common problems, but hard to stop reliably. Better to check-and-bail-out-early across all subclasses.

          I don't want to add a PENDING-SHUTDOWN statehowever, as that just complicates things more

          Show
          Steve Loughran added a comment - First walk through; common problems, but hard to stop reliably. Better to check-and-bail-out-early across all subclasses. I don't want to add a PENDING-SHUTDOWN statehowever, as that just complicates things more
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12505908/MAPREDUCE-3502.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505908/MAPREDUCE-3502.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 14 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1385//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          Improved patch

          1. javadoc of Service interface and AbstractService so as to more clearly specify the desired behaviour of implementations.
          2. common methods in AbstractService to interrupt a non-null thread, stop non-null IPC and webapp servers. These calls return null values to overwrite field variables after the successful operation.
          3. static methods to stop a service if non null, and one to do it with exceptions caught and logged.

          All subclasses have been reviewed and their stop operations

          1. use the new common interrupt and stop methods where appropriate
          2. use checks for null fields before trying any stop operation
          3. nullify all non-final fields after their work.

          Some but not all of the stop() methods join onto the thread that was just interrupted; to wait for it to stop. This would be the cleanest option, as it guarantees the worker threads will not invoke a (now-stopped) service. I have not changed the behaviour of any existing services that do not perform Thread.join() operations. If there was a well defined behaviour for that join (time to wait, exception to throw on failure), it could be injected into the new AbstractService.interruptThread() method, and used throughout the services. Having a consistent interrupt/join/report failure process would imply only one thing to test, and more trust that new services will follow the examples set in the MRv2 codebase.

          This patch does nothing to address the issue of MAPREDUCE-3535 -that the stoppability of a service is not checked before child classes terminate; it is somewhat orthogonal.

          This patch is designed to ensure that no matter when Service.stop() is called (except in the special situation of a re-entrant call on a separate thread), everything is shut down cleanly. This means that even if the failure is triggered halfway through the init(Conf) or start() operations the stop() operation will always clean up what there is.

          Show
          Steve Loughran added a comment - Improved patch javadoc of Service interface and AbstractService so as to more clearly specify the desired behaviour of implementations. common methods in AbstractService to interrupt a non-null thread, stop non-null IPC and webapp servers. These calls return null values to overwrite field variables after the successful operation. static methods to stop a service if non null, and one to do it with exceptions caught and logged. All subclasses have been reviewed and their stop operations use the new common interrupt and stop methods where appropriate use checks for null fields before trying any stop operation nullify all non-final fields after their work. Some but not all of the stop() methods join onto the thread that was just interrupted; to wait for it to stop. This would be the cleanest option, as it guarantees the worker threads will not invoke a (now-stopped) service. I have not changed the behaviour of any existing services that do not perform Thread.join() operations. If there was a well defined behaviour for that join (time to wait, exception to throw on failure), it could be injected into the new AbstractService.interruptThread() method, and used throughout the services. Having a consistent interrupt/join/report failure process would imply only one thing to test, and more trust that new services will follow the examples set in the MRv2 codebase. This patch does nothing to address the issue of MAPREDUCE-3535 -that the stoppability of a service is not checked before child classes terminate; it is somewhat orthogonal. This patch is designed to ensure that no matter when Service.stop() is called (except in the special situation of a re-entrant call on a separate thread), everything is shut down cleanly. This means that even if the failure is triggered halfway through the init(Conf) or start() operations the stop() operation will always clean up what there is.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12507037/MAPREDUCE-3502.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 12 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507037/MAPREDUCE-3502.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 12 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 2 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1422//console This message is automatically generated.
          Hide
          Steve Loughran added a comment -

          Plan
          # submit Service w/ better shutdown (and helper) methods first

          1. submit patches to specific sections of the yarn codebase
          Show
          Steve Loughran added a comment - Plan # submit Service w/ better shutdown (and helper) methods first submit patches to specific sections of the yarn codebase
          Hide
          Steve Loughran added a comment -
          org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher  Time elapsed: 4137 sec  <<< ERROR!
          java.lang.RuntimeException: Error parsing 'yarn-site.xml' : org.xml.sax.SAXParseException: Premature end of file.
          	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2050)
          	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1899)
          	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1816)
          	at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:465)
          	at org.apache.hadoop.conf.Configuration.asXmlDocument(Configuration.java:2127)
          	at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2096)
          	at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2086)
          	at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.setup(TestUnmanagedAMLauncher.java:63)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
          	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
          	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
          	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
          	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
          	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
          Caused by: org.xml.sax.SAXParseException: Premature end of file.
          	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:246)
          	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
          	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:153)
          	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1887)
          	at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1875)
          	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1946)
          	... 29 more
          

          This stack trace is a failure to read the file yarn-site.xml, which is actually being written on line 63 of TestUnmanagedAMLauncher -a file that
          is already open for writing.

          It is possible that some filesystems (here, HFS+) make that write visible while it is still
          going on, triggering a failure which then corrupts later builds at init time

          $ ls -l target/test-classes/yarn-site.xml 
          -rw-r--r--  1 stevel  staff  0  3 Apr 15:37 target/test-classes/yarn-site.xml
          

          This is newer than the one in test/properties, so Maven doesn't fix it next test run

          $ ls -l src/test/resources/yarn-site.xml 
          -rw-r--r--@ 1 stevel  staff  830 28 Nov 16:29 src/test/resources/yarn-site.xml
          

          as a result, follow on tests fail when MiniYARNCluster tries to read it.

          org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher  Time elapsed: 515 sec  <<< ERROR!
          java.lang.RuntimeException: Error parsing 'yarn-site.xml' : org.xml.sax.SAXParseException: Premature end of file.
          	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2050)
          	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1899)
          	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1816)
          	at org.apache.hadoop.conf.Configuration.get(Configuration.java:719)
          	at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:738)
          	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:948)
          	at org.apache.hadoop.security.Groups.<init>(Groups.java:62)
          	at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182)
          	at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:244)
          	at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:213)
          	at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:674)
          	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:582)
          	at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:458)
          	at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:439)
          	at org.apache.hadoop.fs.FileContext.getLocalFSFileContext(FileContext.java:425)
          	at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:93)
          	at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.setup(TestUnmanagedAMLauncher.java:52)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
          	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
          
          1. This test should copy over the new yarn-site.xml rather than overwrite the live one
          2. there should be a yarn-site-template.xml that is copied over
          3. the Configuration.loadResource() could skip size 0 files instead of trying to parse them.
          Show
          Steve Loughran added a comment - org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher Time elapsed: 4137 sec <<< ERROR! java.lang.RuntimeException: Error parsing 'yarn-site.xml' : org.xml.sax.SAXParseException: Premature end of file. at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2050) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1899) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1816) at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:465) at org.apache.hadoop.conf.Configuration.asXmlDocument(Configuration.java:2127) at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2096) at org.apache.hadoop.conf.Configuration.writeXml(Configuration.java:2086) at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.setup(TestUnmanagedAMLauncher.java:63) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: org.xml.sax.SAXParseException: Premature end of file. at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:246) at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:153) at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1887) at org.apache.hadoop.conf.Configuration.parse(Configuration.java:1875) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1946) ... 29 more This stack trace is a failure to read the file yarn-site.xml, which is actually being written on line 63 of TestUnmanagedAMLauncher -a file that is already open for writing. It is possible that some filesystems (here, HFS+) make that write visible while it is still going on, triggering a failure which then corrupts later builds at init time $ ls -l target/test-classes/yarn-site.xml -rw-r--r-- 1 stevel staff 0 3 Apr 15:37 target/test-classes/yarn-site.xml This is newer than the one in test/properties, so Maven doesn't fix it next test run $ ls -l src/test/resources/yarn-site.xml -rw-r--r--@ 1 stevel staff 830 28 Nov 16:29 src/test/resources/yarn-site.xml as a result, follow on tests fail when MiniYARNCluster tries to read it. org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher Time elapsed: 515 sec <<< ERROR! java.lang.RuntimeException: Error parsing 'yarn-site.xml' : org.xml.sax.SAXParseException: Premature end of file. at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2050) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1899) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1816) at org.apache.hadoop.conf.Configuration.get(Configuration.java:719) at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:738) at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:948) at org.apache.hadoop.security.Groups.<init>(Groups.java:62) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:182) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:244) at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:213) at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:674) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:582) at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:458) at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:439) at org.apache.hadoop.fs.FileContext.getLocalFSFileContext(FileContext.java:425) at org.apache.hadoop.yarn.server.MiniYARNCluster.<init>(MiniYARNCluster.java:93) at org.apache.hadoop.yarn.applications.unmanagedamlauncher.TestUnmanagedAMLauncher.setup(TestUnmanagedAMLauncher.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) This test should copy over the new yarn-site.xml rather than overwrite the live one there should be a yarn-site-template.xml that is copied over the Configuration.loadResource() could skip size 0 files instead of trying to parse them.
          Hide
          Steve Loughran added a comment -

          ignore that last comment pasted into wrong issue

          Show
          Steve Loughran added a comment - ignore that last comment pasted into wrong issue
          Hide
          Steve Loughran added a comment -

          this is fixed now

          Show
          Steve Loughran added a comment - this is fixed now

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Time Spent - 2.5h Remaining Estimate - 21.5h
                21.5h
                Logged:
                Time Spent - 2.5h Remaining Estimate - 21.5h
                2.5h

                  Development