Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-9115

Command failed with error 17419: 'Resulting document after update is larger than 16777216'

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.8.22
    • Fix Version/s: None
    • Component/s: mongomk
    • Labels:
      None
    • Environment:

      Oak: 1.8.22
      Mongo Node Store
      S3 Blob Store

      Description

      When attempting to save a new item of content into Oak, we are now getting the following message. As mentioned in a previous issue raised (OAK-9112), we have recently backloaded a lot of content into Oak with the Mongo "nodes" collection increasing from 100,000 documents to 600,000 documents.

      The new content we are creating is less than 1k in size.

      I appreciate this probably not a bug but any guidance resolving this issue in a production system would be appreciated.

      We can update existing content items without issue only creating new items is causing the stack trace below.

      16 Jun 2020 08:03:34.263HostName=crms-service-content-f899d5f6d-lwmgp 08:03:34.223 [content-service] [http-nio-8080-exec-195] ERROR o.s.c.s.instrument.web.TraceFilter - Uncaught exception thrown
      org.springframework.web.util.NestedServletException: Request processing failed; nested exception is javax.jcr.InvalidItemStateException: OakMerge0001: OakMerge0001: Failed to merge changes to the underlying store (retries 5, 70255 ms)
      
      15 Jun 2020 19:07:33.228HostName=crms-service-content-f899d5f6d-qr9lt 19:07:33.126 [content-service] [http-nio-8080-exec-311] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is javax.jcr.InvalidItemStateException: OakMerge0001: OakMerge0001: Failed to merge changes to the underlying store (retries 5, 68221 ms)] with root cause
      com.mongodb.MongoCommandException: Command failed with error 17419: 'Resulting document after update is larger than 16777216' on server sg-pbprod-10448.servers.mongodirector.com:27017. The full response is { "ok" : 0.0, "errmsg" : "Resulting document after update is larger than 16777216", "code" : 17419, "codeName" : "Location17419" }
      	at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
      	at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)
      	at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
      	at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)
      	at com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:176)
      	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:216)
      	at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:153)
      	at com.mongodb.operation.FindAndUpdateOperation$1.call(FindAndUpdateOperation.java:335)
      	at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:426)
      	at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:417)
      	at com.mongodb.operation.FindAndUpdateOperation.execute(FindAndUpdateOperation.java:331)
      	at com.mongodb.Mongo.execute(Mongo.java:845)
      	at com.mongodb.Mongo$2.execute(Mongo.java:828)
      	at com.mongodb.DBCollection.findAndModify(DBCollection.java:1928)
      	at com.mongodb.DBCollection.findAndModify(DBCollection.java:1792)
      	at com.mongodb.DBCollection.findAndModify(DBCollection.java:1761)
      	at com.mongodb.DBCollection.findAndModify(DBCollection.java:1708)
      	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:862)
      	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:908)
      	at org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:987)
      	at org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.createOrUpdate(LeaseCheckDocumentStoreWrapper.java:136)
      	at org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:343)
      	at org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:264)
      	at org.apache.jackrabbit.oak.plugins.document.Commit.applyInternal(Commit.java:233)
      	at org.apache.jackrabbit.oak.plugins.document.Commit.apply(Commit.java:222)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:301)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:271)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.access$400(DocumentNodeStoreBranch.java:57)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$InMemory.merge(DocumentNodeStoreBranch.java:519)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge0(DocumentNodeStoreBranch.java:188)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge(DocumentNodeStoreBranch.java:130)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentRootBuilder.merge(DocumentRootBuilder.java:163)
      	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.merge(DocumentNodeStore.java:1914)
      	at org.apache.jackrabbit.oak.core.MutableRoot.commit(MutableRoot.java:250)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.commit(SessionDelegate.java:346)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:493)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:424)
      	at org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
      	at org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:421)
      	at au.com.calibreft.service.content.dao.JcrRepository.executeWithSession(JcrRepository.java:527)
      	at au.com.calibreft.service.content.dao.JcrRepository.executeSaveWithSession(JcrRepository.java:494)
      	at au.com.calibreft.service.content.dao.JcrRepository.save(JcrRepository.java:472)
      	at au.com.calibreft.service.content.dao.JcrRepository$$FastClassBySpringCGLIB$$b81905ec.invoke(<generated>)
      	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
      	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
      	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
      	at au.com.calibreft.service.content.dao.JcrRepository$$EnhancerBySpringCGLIB$$a5620910.save(<generated>)
      	at au.com.calibreft.service.content.rest.ContentController.saveContent(ContentController.java:126)
      	at sun.reflect.GeneratedMethodAccessor286.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
      	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
      	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
      	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:854)
      	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:765)
      	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
      	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
      	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
      	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
      	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
      	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:54)
      	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              djsnowsill David Snowsill
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: