Uploaded image for project: 'Jackrabbit Content Repository'
  1. Jackrabbit Content Repository
  2. JCR-3873

CachingDataStore not safe against crashes, corrupted uploads file will prevent system startup

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.9.1
    • 2.10.1
    • jackrabbit-data
    • None

    Description

      An NPE in OAK-2811 lead to the data store not being closed normally on shutdown, in turn leaving the async-pending-uploads.ser file corrupted (on this system, binaries were constantly uploaded, so likely the queue to upload it to S3 in this case was still full) and then on the next startup, the CachingDataStore failed to start up, preventing the repository and the system from starting:

      24.04.2015 13:21:28.602 *ERROR* [FelixStartLevel] org.apache.jackrabbit.oak-core [org.apache.jackrabbit.oak.plugins.blob.datastore.S3DataStore(79)] The activate method has thrown an exception (javax.jcr.RepositoryException: java.io.EOFException)
      javax.jcr.RepositoryException: java.io.EOFException
      	at org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:337)
      	at org.apache.jackrabbit.oak.plugins.blob.datastore.AbstractDataStoreService.activate(AbstractDataStoreService.java:61)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
      	at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
      	at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
      	at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
      	at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
      	at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
      	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:399)
      	at org.apache.felix.scr.impl.config.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:669)
      	at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:184)
      	at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:332)
      	at org.apache.felix.scr.impl.Activator.access$000(Activator.java:49)
      	at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:257)
      	at org.apache.felix.utils.extender.AbstractExtender.createExtension(AbstractExtender.java:259)
      	at org.apache.felix.utils.extender.AbstractExtender.modifiedBundle(AbstractExtender.java:232)
      	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:482)
      	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:415)
      	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
      	at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:444)
      	at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:913)
      	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
      	at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
      	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4531)
      	at org.apache.felix.framework.Felix.startBundle(Felix.java:2169)
      	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1368)
      	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:308)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.io.EOFException: null
      	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
      	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
      	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
      	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
      	at org.apache.jackrabbit.core.data.AsyncUploadCache.deserializeAsyncUploadMap(AsyncUploadCache.java:311)
      	at org.apache.jackrabbit.core.data.AsyncUploadCache.init(AsyncUploadCache.java:243)
      	at org.apache.jackrabbit.core.data.CachingDataStore.init(CachingDataStore.java:264)
      	... 35 common frames omitted
      

      We had to manually remove the file so that the system would start again. AFAIU, this meant some data loss.

      Attachments

        Activity

          People

            shgupta Shashank Gupta
            alexander.klimetschek Alexander Klimetschek
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: