Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7489

ConcurrentModificationException in RMAppImpl#getRMAppMetrics

    Details

      Description

      The REST clients have sometimes failed to query applications through apps REST API in RMWebService and it happened when iterating attempts(RMWebServices#getApps --> AppInfo#<init> --> RMAppImpl#getRMAppMetrics) and meanwhile these attempts changed(AttemptFailedTransition#transition --> RMAppImpl#createAndStartNewAttempt --> RMAppImpl#createNewAttempt).
      Application state changed within the lockup period of writeLock in RMAppImpl, so that we can add readLock before iterating attempts to fix this problem.
      Exception stack:

      java.util.ConcurrentModificationException
              at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719)
              at java.util.LinkedHashMap$LinkedValueIterator.next(LinkedHashMap.java:747)
              at org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl.getRMAppMetrics(RMAppImpl.java:1487)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.AppInfo.<init>(AppInfo.java:199)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getApps(RMWebServices.java:597)
              at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
              at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
              at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
              at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
              at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
              at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
              at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
              at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
              at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
              at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
              at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
              at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
              at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
              at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
              at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:886)
              at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
              at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:178)
              at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
              at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
              at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
              at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
      

        Attachments

        1. YARN-7489.001.patch
          4 kB
          Tao Yang
        2. YARN-7489-branch-2.001.patch
          3 kB
          Tao Yang

          Activity

            People

            • Assignee:
              Tao Yang Tao Yang
              Reporter:
              Tao Yang Tao Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: