Uploaded image for project: 'OpenWebBeans'
  1. OpenWebBeans
  2. OWB-1123

NPE at NormalScopeProxyFactory.createNormalScopeProxy during deserialization

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None
    • Environment:
      TomEE 7.0.0.M2

      Description

      Imagine a session scoped bean which has an application scoped bean injected as property and it's being referenced in readObject().

      @SessionScoped
      public class SessionScopedBean {
      
          @Inject
          private ApplicationScopedBean appBean;
      
          private void readObject(ObjectInputStream input) throws IOException, ClassNotFoundException {
              appBean.doSomething();
          }
      }
      

      OWB 1.6.2 as used in TomEE 7.0.0.M2 throws below exception when TomEE is restarted with session persistence (and above bean is created during that session):

      java.lang.NullPointerException
          at org.apache.webbeans.proxy.NormalScopeProxyFactory.createNormalScopeProxy(NormalScopeProxyFactory.java:121)
          at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:125)
      

      It works fine in all Weld 2.x versions I tested so far (WF8/9/10, PY4 and "plain" Tomcat8+Weld 2.3.0).

      Work around would be to use a static method on application scoped bean, but this is not nice.

        Activity

        Hide
        struberg Mark Struberg added a comment -

        shipped with OWB-1.7.0

        Show
        struberg Mark Struberg added a comment - shipped with OWB-1 .7.0
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        cdi-tomee is green to so change works

        In summarize we need a cluster stable generated id for annotatedType (propagatedto bean PassivationCapable id as describe by extension API). Issue was ext.toString() was used and we now use ext.class which is stable. We need that prefixing cause several extensions can register the same bean multiple times and we need to be able to distinguish them - tested in TCK IIRC.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - cdi-tomee is green to so change works In summarize we need a cluster stable generated id for annotatedType (propagatedto bean PassivationCapable id as describe by extension API). Issue was ext.toString() was used and we now use ext.class which is stable. We need that prefixing cause several extensions can register the same bean multiple times and we need to be able to distinguish them - tested in TCK IIRC.
        Hide
        struberg Mark Struberg added a comment -

        In fact it should be:

        • AnnotatedTypeWrapper(final Extension source, final AnnotatedType<T> annotatedType)
          + AnnotatedTypeWrapper(final Class<? extends Extension> source, final AnnotatedType<T> annotatedType)

        That way it's perfectly serializable. Using toString is too fragile imo.
        Otoh I have no clue why the AnnotatedType gets stored in MyFaces. Didn't read the whole thread in detail though.

        Show
        struberg Mark Struberg added a comment - In fact it should be: AnnotatedTypeWrapper(final Extension source, final AnnotatedType<T> annotatedType) + AnnotatedTypeWrapper(final Class<? extends Extension> source, final AnnotatedType<T> annotatedType) That way it's perfectly serializable. Using toString is too fragile imo. Otoh I have no clue why the AnnotatedType gets stored in MyFaces. Didn't read the whole thread in detail though.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        cdi-embedded (tomee embedded tck) are green, pushed a fix using the extension classname instead of the tostring value.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - cdi-embedded (tomee embedded tck) are green, pushed a fix using the extension classname instead of the tostring value.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment - - edited

        Ok, the cause is there https://github.com/apache/openwebbeans/blob/trunk/webbeans-impl/src/main/java/org/apache/webbeans/container/BeanManagerImpl.java#L1278 (not the extension.toString()). Fix can be in both OWB and myfaces. On MyFaces side the extension could implement a stable toString() (workaround) but should set the annotated type id manually (recommanded way). On OWB we can evaluate the fact to not rely on toString there but we need to pass TCKs first (issue being how to distinguish 2 extensions of the same type in an ear for instance).

        edit: running tck ATM (and off to breakfast), will come back there soon to give a status on that

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - - edited Ok, the cause is there https://github.com/apache/openwebbeans/blob/trunk/webbeans-impl/src/main/java/org/apache/webbeans/container/BeanManagerImpl.java#L1278 (not the extension.toString()). Fix can be in both OWB and myfaces. On MyFaces side the extension could implement a stable toString() (workaround) but should set the annotated type id manually (recommanded way). On OWB we can evaluate the fact to not rely on toString there but we need to pass TCKs first (issue being how to distinguish 2 extensions of the same type in an ear for instance). edit: running tck ATM (and off to breakfast), will come back there soon to give a status on that
        Hide
        lu4242 Leonardo Uribe added a comment -

        It should be possible to reproduce the problem without JSF

        Show
        lu4242 Leonardo Uribe added a comment - It should be possible to reproduce the problem without JSF
        Hide
        lu4242 Leonardo Uribe added a comment - - edited

        I can see that org.apache.myfaces.cdi.view.ViewScopeBeanHolder is the session scope bean that has the reference. The problem is in OWB, the use of @ViewScoped trigger the use of this bean (because this bean is the one that holds the view scope bean info), but if the deserialization happens on the right place, ApplicationContextBean should be available. The problem I see is the deserialisation step could happen outside CDI control, and OWB must ensure that case to work (the bean in that case must be resolved in a lazy way). In previous OWB versions, there was a wrapper over the beans, and I guess that wrapper had a logic to avoid that case.

        Show
        lu4242 Leonardo Uribe added a comment - - edited I can see that org.apache.myfaces.cdi.view.ViewScopeBeanHolder is the session scope bean that has the reference. The problem is in OWB, the use of @ViewScoped trigger the use of this bean (because this bean is the one that holds the view scope bean info), but if the deserialization happens on the right place, ApplicationContextBean should be available. The problem I see is the deserialisation step could happen outside CDI control, and OWB must ensure that case to work (the bean in that case must be resolved in a lazy way). In previous OWB versions, there was a wrapper over the beans, and I guess that wrapper had a logic to avoid that case.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Adam Cornett sounds like another issue. Would also need an arquillian test (you can define a cluster) to reproduce it but looks like some bean id instability using a hashcode

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Adam Cornett sounds like another issue. Would also need an arquillian test (you can define a cluster) to reproduce it but looks like some bean id instability using a hashcode
        Hide
        acornett Adam Cornett added a comment -

        Versions (snapshots pulled down at Aug 24 20:30 UTC)

        • TomEE Plus 7.0.2-SNAPSHOT
        • OWB 1.6.4-SNAPSHOT
        • MyFaces 2.2.11-SNAPSHOT

        I have native tomcat clustering/session replication setup and when accessing a page with a @ViewScoped bean I get the following error on the system which is receiving the session update:

        2016-08-24 21:43:02.322 [Tribes-Task-Receiver[test.local-Channel]-6] ERROR org.apache.catalina.ha.session.DeltaManager - Manager []: Unable to receive message through TCP channel
        java.io.NotSerializableException: Failure during de-serialisation: Cannot load Bean with passivationId org.apache.myfaces.cdi.view.ViewScopeContextExtension@71bc6914Base Type : class org.apache.myfaces.cdi.view.
        ApplicationContextBean,Type Closures : null,Annotations : [@javax.enterprise.context.ApplicationScoped()]OWB_DEFAULT_KEY
                at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:127)
                at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148)
                at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1817)
                at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
                at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
                at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
                at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
                at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
                at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
                at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
                at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
                at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
                at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
                at java.util.HashMap.readObject(HashMap.java:1404)
                at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
                at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
                at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
                at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
                at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
                at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54)
                at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1849)
                at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1806)
                at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
                at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
                at org.apache.catalina.ha.session.DeltaRequest$AttributeInfo.readExternal(DeltaRequest.java:387)
                at org.apache.catalina.ha.session.DeltaRequest.readExternal(DeltaRequest.java:277)
                at org.apache.catalina.ha.session.DeltaManager.deserializeDeltaRequest(DeltaManager.java:573)
                at org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA(DeltaManager.java:1245)
                at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1189)
                at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:929)
                at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:77)
                at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:783)
                at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:764)
                at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:300)
                at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83)
                at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83)
                at org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.messageReceived(ThroughputInterceptor.java:91)
                at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83)
                at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:276)
                at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:244)
                at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:213)
                at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:101)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                at java.lang.Thread.run(Thread.java:745)
        

        So it looks like the @ApplicationScoped MyFaces ApplicationContextBean isn't showing up in the passivationBeans map of the remote BeanManagerImpl when OWB is trying to de-serialize the incoming data.
        I'd be glad to provide more testing assistance or try patches if needed.

        Show
        acornett Adam Cornett added a comment - Versions (snapshots pulled down at Aug 24 20:30 UTC) TomEE Plus 7.0.2-SNAPSHOT OWB 1.6.4-SNAPSHOT MyFaces 2.2.11-SNAPSHOT I have native tomcat clustering/session replication setup and when accessing a page with a @ViewScoped bean I get the following error on the system which is receiving the session update: 2016-08-24 21:43:02.322 [Tribes-Task-Receiver[test.local-Channel]-6] ERROR org.apache.catalina.ha.session.DeltaManager - Manager []: Unable to receive message through TCP channel java.io.NotSerializableException: Failure during de-serialisation: Cannot load Bean with passivationId org.apache.myfaces.cdi.view.ViewScopeContextExtension@71bc6914Base Type : class org.apache.myfaces.cdi.view. ApplicationContextBean,Type Closures : null,Annotations : [@javax.enterprise.context.ApplicationScoped()]OWB_DEFAULT_KEY at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:127) at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1817) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) at java.util.HashMap.readObject(HashMap.java:1404) at sun.reflect.GeneratedMethodAccessor156.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54) at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1849) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1806) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) at org.apache.catalina.ha.session.DeltaRequest$AttributeInfo.readExternal(DeltaRequest.java:387) at org.apache.catalina.ha.session.DeltaRequest.readExternal(DeltaRequest.java:277) at org.apache.catalina.ha.session.DeltaManager.deserializeDeltaRequest(DeltaManager.java:573) at org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA(DeltaManager.java:1245) at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1189) at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:929) at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:77) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:783) at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:764) at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:300) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83) at org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.messageReceived(ThroughputInterceptor.java:91) at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:83) at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:276) at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:244) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:213) at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:101) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) So it looks like the @ApplicationScoped MyFaces ApplicationContextBean isn't showing up in the passivationBeans map of the remote BeanManagerImpl when OWB is trying to de-serialize the incoming data. I'd be glad to provide more testing assistance or try patches if needed.
        Hide
        lu4242 Leonardo Uribe added a comment -

        I see. I have tested already if ViewScopeContextualStorage can be serialized and I can't see any problem there. MyFaces uses a CDI session scope bean to hold the view scopes (ViewScopeBeanHolder).

        I think it should be some instance that is not serializable inside ViewScopeContextualStorage but that comes from CDI. With the previous method, I guess everything was packed somehow to be serialized later. If that instance is not serializable, it is just skipped on serialization but it lets a null on deserialization and throw the null pointer exception. There is a call to java.util.HashMap.readObject(...) in the stack trace, so it is something inside:

        private final Map<Object, ContextualInstanceInfo<?>> contextualInstances;

        or

        private final Map<String, Object> nameBeanKeyMap;

        Show
        lu4242 Leonardo Uribe added a comment - I see. I have tested already if ViewScopeContextualStorage can be serialized and I can't see any problem there. MyFaces uses a CDI session scope bean to hold the view scopes (ViewScopeBeanHolder). I think it should be some instance that is not serializable inside ViewScopeContextualStorage but that comes from CDI. With the previous method, I guess everything was packed somehow to be serialized later. If that instance is not serializable, it is just skipped on serialization but it lets a null on deserialization and throw the null pointer exception. There is a call to java.util.HashMap.readObject(...) in the stack trace, so it is something inside: private final Map<Object, ContextualInstanceInfo<?>> contextualInstances; or private final Map<String, Object> nameBeanKeyMap;
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Leonardo Uribe just as a FYI: OWB now stores @SessionScoped instances in the HTTP session where before it was matching them based on the session id in a local map. This is the change which triggered this jira related behavior.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Leonardo Uribe just as a FYI: OWB now stores @SessionScoped instances in the HTTP session where before it was matching them based on the session id in a local map. This is the change which triggered this jira related behavior.
        Hide
        lu4242 Leonardo Uribe added a comment -

        I have checked the example reported in MyFaces side and the problem is related to some changes in OWB. In TomEE 1.7.4 (OWB 1.2.7) it works well but in TomEE 7.0.1 (OWB 1.6.3) it throws the exception.

        It is clear rely on BeanManager serialization is not a good idea, so I fixed the code to avoid the serialization step and get the BeanManager using FacesContext, but that doesn't fix the exception at restart (it fix the one at shutdown). So I guess it should be something wrong in OWB 1.6.2 and upper versions. I can't see anything different on the stack trace.

        Show
        lu4242 Leonardo Uribe added a comment - I have checked the example reported in MyFaces side and the problem is related to some changes in OWB. In TomEE 1.7.4 (OWB 1.2.7) it works well but in TomEE 7.0.1 (OWB 1.6.3) it throws the exception. It is clear rely on BeanManager serialization is not a good idea, so I fixed the code to avoid the serialization step and get the BeanManager using FacesContext, but that doesn't fix the exception at restart (it fix the one at shutdown). So I guess it should be something wrong in OWB 1.6.2 and upper versions. I can't see anything different on the stack trace.
        Hide
        struberg Mark Struberg added a comment -

        Meanwhile I improved the error reporting on our side and intend to close this issue. Is this fine for you?

        Show
        struberg Mark Struberg added a comment - Meanwhile I improved the error reporting on our side and intend to close this issue. Is this fine for you?
        Hide
        balusc Bauke Scholtz added a comment -
        Show
        balusc Bauke Scholtz added a comment - Okay, I reported over there: https://issues.apache.org/jira/browse/MYFACES-4057
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Sounds consistent with my tests. Just wonder if we can do something about that. There is an InjectableBeanManager which should be the one used by myfaces which is serializable, maybe they capture it wrong or we dont expose it the way they expect.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Sounds consistent with my tests. Just wonder if we can do something about that. There is an InjectableBeanManager which should be the one used by myfaces which is serializable, maybe they capture it wrong or we dont expose it the way they expect.
        Hide
        balusc Bauke Scholtz added a comment - - edited

        I've finally nailed it. After all, it's not caused by OWB, but by MyFaces. Its org.apache.myfaces.cdi.view.ViewScopeContextualStorage is stored in session, but it has a non-transient BeanManager property. The BeanManager interface is not declared to be Serializable. In Weld, the BeanManagerImpl is Serializable, but in OWB this is not the case. So I gather this is a bug in MyFaces. If you agree this is a bug in MyFaces, I will report over there.

        I've boiled down the smallest possible reproducer: just have a @Named @javax.faces.view.ViewScoped bean and then open a JSF page referencing it and then restart the server.

        @Named
        @ViewScoped
        public class Bean implements Serializable {}
        
        <!DOCTYPE html>
        #{bean}
        

        Sorry for the wrong initial issue. I was being misled because the actual @SessionScoped bean implementation in turn touches the JSF view scope and the stack trace didn't give any clues as to the root cause.

        Show
        balusc Bauke Scholtz added a comment - - edited I've finally nailed it. After all, it's not caused by OWB, but by MyFaces. Its org.apache.myfaces.cdi.view.ViewScopeContextualStorage is stored in session, but it has a non-transient BeanManager property. The BeanManager interface is not declared to be Serializable. In Weld, the BeanManagerImpl is Serializable, but in OWB this is not the case. So I gather this is a bug in MyFaces. If you agree this is a bug in MyFaces, I will report over there. I've boiled down the smallest possible reproducer: just have a @Named @javax.faces.view.ViewScoped bean and then open a JSF page referencing it and then restart the server. @Named @ViewScoped public class Bean implements Serializable {} <!DOCTYPE html> #{bean} Sorry for the wrong initial issue. I was being misled because the actual @SessionScoped bean implementation in turn touches the JSF view scope and the stack trace didn't give any clues as to the root cause.
        Hide
        struberg Mark Struberg added a comment -

        Can you probably debug into and check what Object causes the problem? txs!

        Show
        struberg Mark Struberg added a comment - Can you probably debug into and check what Object causes the problem? txs!
        Hide
        balusc Bauke Scholtz added a comment -

        When digging further, the root cause appears to be the below which got logged when TomEE was shutdown instead of restarted. When TomEE was restarted, this warning goes unnoticed because the IDE clears out the entire console.

        WARNING: Cannot serialize session attribute openWebBeansSessionContext for session 34F40C623600849F43AD2153F2EB75D8
        java.io.NotSerializableException: org.apache.webbeans.container.BeanManagerImpl
        	at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
        

        When above is logged on shutdown, then the startup will consistently throw NPE on createNormalScopeProxy. I will this week try to create a reproducer.

        Show
        balusc Bauke Scholtz added a comment - When digging further, the root cause appears to be the below which got logged when TomEE was shutdown instead of restarted. When TomEE was restarted, this warning goes unnoticed because the IDE clears out the entire console. WARNING: Cannot serialize session attribute openWebBeansSessionContext for session 34F40C623600849F43AD2153F2EB75D8 java.io.NotSerializableException: org.apache.webbeans.container.BeanManagerImpl at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) When above is logged on shutdown, then the startup will consistently throw NPE on createNormalScopeProxy. I will this week try to create a reproducer.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Added a test on OWB and tested on TomEE 7.0.1 and the issue is not relevant for built-in session managers so really looks like an issue on memcache impl side.

        If you can edit the test case to make it failling we can move forward on this topic. Here it is https://github.com/apache/openwebbeans/blob/trunk/webbeans-tomcat7/src/test/java/org/apache/webbeans/web/tomcat7/test/TomcatNormalScopeProxyFactoryTest.java

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Added a test on OWB and tested on TomEE 7.0.1 and the issue is not relevant for built-in session managers so really looks like an issue on memcache impl side. If you can edit the test case to make it failling we can move forward on this topic. Here it is https://github.com/apache/openwebbeans/blob/trunk/webbeans-tomcat7/src/test/java/org/apache/webbeans/web/tomcat7/test/TomcatNormalScopeProxyFactoryTest.java
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Should be fixed next week

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Should be fixed next week
        Hide
        hurrrrricaaaaane Josef Puff added a comment -

        Any news here?

        Show
        hurrrrricaaaaane Josef Puff added a comment - Any news here?
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        [to give more details cause Mark said me I was unclear] Tomcat load/unload sessions mainly at startup/shutdown times. At that time (startup is the one we care) CDI doesn't exist yet so if any Bean<?> instance is serialized at shutdown it will be missing at startup. For passivation capable normal scope instancesit is not that important in general since they have a particular handling and they can just be ignored in most cases (what does the proposed listener). Hope it is clearer.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - [to give more details cause Mark said me I was unclear] Tomcat load/unload sessions mainly at startup/shutdown times. At that time (startup is the one we care) CDI doesn't exist yet so if any Bean<?> instance is serialized at shutdown it will be missing at startup. For passivation capable normal scope instancesit is not that important in general since they have a particular handling and they can just be ignored in most cases (what does the proposed listener). Hope it is clearer.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Checking MemcachedBackupSessionManager it seems a memcached implementation issue (load/unload are skipped so session are not cleaned up at shutdown time). This listener fixes it for StandardSession of tomcat (default) but memcached should be enhanced I think to support this use case - hope i didnt look too fast on github.

        Alternative is to keep track of all actives sessions - with a listener - to execute this removeAttribute in a ServletContextListener on destroy phase).

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Checking MemcachedBackupSessionManager it seems a memcached implementation issue (load/unload are skipped so session are not cleaned up at shutdown time). This listener fixes it for StandardSession of tomcat (default) but memcached should be enhanced I think to support this use case - hope i didnt look too fast on github. Alternative is to keep track of all actives sessions - with a listener - to execute this removeAttribute in a ServletContextListener on destroy phase).
        Hide
        hurrrrricaaaaane Josef Puff added a comment - - edited

        Okay thanks.

        I tried it bot the same exception is in the log.

        Web.xml

        <listener>
        	<listener-class>com.applicationName.appName.web.internal.cluster.RemoveCdiBeansListener</listener-class>
        </listener>
        
        Show
        hurrrrricaaaaane Josef Puff added a comment - - edited Okay thanks. I tried it bot the same exception is in the log. Web.xml <listener> <listener-class> com.applicationName.appName.web.internal.cluster.RemoveCdiBeansListener </listener-class> </listener>
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        should be register as a listener (@WebListener is a way or web.xml)

        If it doesn't work with memcache session manager then memcache session manager has a bug I think cause will passivate should be called at shutdown

        Side note: this supposes the shutdown was clean so ensure to remove previous sessions for the first try

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - should be register as a listener (@WebListener is a way or web.xml) If it doesn't work with memcache session manager then memcache session manager has a bug I think cause will passivate should be called at shutdown Side note: this supposes the shutdown was clean so ensure to remove previous sessions for the first try
        Hide
        hurrrrricaaaaane Josef Puff added a comment -

        I'm not sure where I should implement this snippet.
        I added this snippet in out "common" package. Is that correct?

        The test with this code wasn't successful.

        Show
        hurrrrricaaaaane Josef Puff added a comment - I'm not sure where I should implement this snippet. I added this snippet in out "common" package. Is that correct? The test with this code wasn't successful.
        Hide
        hurrrrricaaaaane Josef Puff added a comment -

        Romain Manni-Bucau Thanks I'll try it.
        Mark Struberg Oh this would be very nice!

        Show
        hurrrrricaaaaane Josef Puff added a comment - Romain Manni-Bucau Thanks I'll try it. Mark Struberg Oh this would be very nice!
        Hide
        struberg Mark Struberg added a comment -

        I'll try to get it done in the next 2 days.

        Show
        struberg Mark Struberg added a comment - I'll try to get it done in the next 2 days.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment - - edited
        import javax.servlet.http.HttpSessionActivationListener;
        import javax.servlet.http.HttpSessionEvent;
        
        public class RemoveCdiBeansListener implements HttpSessionActivationListener {
            @Override
            public void sessionWillPassivate(final HttpSessionEvent se) {
                se.getSession().removeAttribute("openWebBeansSessionContext");
            }
        
            @Override
            public void sessionDidActivate(final HttpSessionEvent se) {
                // no-op
            }
        }
        

        should be a working workaround if you want to give it a try

        edit: this has been pushed to tomee master with tomee.session.remove-cdi-beans-on-passivate flag in openwebbeans config to deactivate it if desired

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - - edited import javax.servlet.http.HttpSessionActivationListener; import javax.servlet.http.HttpSessionEvent; public class RemoveCdiBeansListener implements HttpSessionActivationListener { @Override public void sessionWillPassivate( final HttpSessionEvent se) { se.getSession().removeAttribute( "openWebBeansSessionContext" ); } @Override public void sessionDidActivate( final HttpSessionEvent se) { // no-op } } should be a working workaround if you want to give it a try edit: this has been pushed to tomee master with tomee.session.remove-cdi-beans-on-passivate flag in openwebbeans config to deactivate it if desired
        Hide
        hurrrrricaaaaane Josef Puff added a comment -

        Okay thanks for fast responsing.
        Do you have any idea for a workaround?

        Is there any plan when this task will be fixed?

        Show
        hurrrrricaaaaane Josef Puff added a comment - Okay thanks for fast responsing. Do you have any idea for a workaround? Is there any plan when this task will be fixed?
        Hide
        tandraschko Thomas Andraschko added a comment - - edited

        Thats a bug for 100% as i'm (almost) sure in the case of Josef, no readObject is used - as i'm coded many parts of the application some years ago.
        Therefore something is probably generally wrong with session persistence.
        I fixed many bugs regarding session replication/persistence in the past and it's a pitty that it fails again.
        We had also had some unit tests, also directly for serialization, in the clustering module.

        Show
        tandraschko Thomas Andraschko added a comment - - edited Thats a bug for 100% as i'm (almost) sure in the case of Josef, no readObject is used - as i'm coded many parts of the application some years ago. Therefore something is probably generally wrong with session persistence. I fixed many bugs regarding session replication/persistence in the past and it's a pitty that it fails again. We had also had some unit tests, also directly for serialization, in the clustering module.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Works with weld cause weld doesn't support bean session clustering (at least by default)

        An easy workaround is to destroy application scoped beans with the application (AlterableContext.class.cast(beamManager.getContext(ApplicationScoped.class)).destroy(bean))

        On OWB side we can surely enhance it to recreate the beans on the fly if it fails there - wouldn't hurt much.

        Another alternative is to filter these OWB beans in the (Memcached)SessionService to ignore them.
        Last alternative is to enforce these beans to be created eagerly (before any request).

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Works with weld cause weld doesn't support bean session clustering (at least by default) An easy workaround is to destroy application scoped beans with the application (AlterableContext.class.cast(beamManager.getContext(ApplicationScoped.class)).destroy(bean)) On OWB side we can surely enhance it to recreate the beans on the fly if it fails there - wouldn't hurt much. Another alternative is to filter these OWB beans in the (Memcached)SessionService to ignore them. Last alternative is to enforce these beans to be created eagerly (before any request).
        Hide
        balusc Bauke Scholtz added a comment - - edited

        Yes that's it, you reuse an old bean.

        Old proxy, you mean? Application scoped bean itself is actually not persisted but just freshly recreated. The proxy is supposed to delegate to the one currently available, not to remember the one from a previous application scope.

        Just disable session persistence then.

        That's absolutely no option. Clustering e.g. would cease to work properly.

        Why does it work perfectly fine and natural in Weld then?

        Show
        balusc Bauke Scholtz added a comment - - edited Yes that's it, you reuse an old bean. Old proxy, you mean? Application scoped bean itself is actually not persisted but just freshly recreated. The proxy is supposed to delegate to the one currently available, not to remember the one from a previous application scope. Just disable session persistence then. That's absolutely no option. Clustering e.g. would cease to work properly. Why does it work perfectly fine and natural in Weld then?
        Hide
        hurrrrricaaaaane Josef Puff added a comment -

        I've the maybe the same issue.
        In my case I have two different Windows Server 2008 R2 with JDK 1.8.0_XX.
        On both there is running a Tomcat 8.0.30 with the following server.xml configuration:

        <?xml version='1.0' encoding='utf-8'?>
        <Server port="8005" shutdown="SHUTDOWN">
          <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
          <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
          <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
          <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
          <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
          
          <Service name="Catalina">
            <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" />
        
            <Connector port="8009" enableLookups="false" 
        	redirectPort="8443" protocol="AJP/1.3" 
        	backlog="50" maxThreads="500" minSpareThreads="25"
        			maxSpareThreads="50" tcpNoDelay="true" soTimeout="60000" 
        			tomcatAuthentication="false" />
        
            <Engine name="Catalina" defaultHost="asmqa.tradeplace.com">
              <Host name="asmqa.tradeplace.com" appBase="D:/WWW/applicationName/webapps" unpackWARs="true" autoDeploy="false" deployOnStartup="false">
        		<Context path="/tp4-web" docBase="D:/WWW/applicationName/webapps/applicationName.war" reloadable="false" unpackWAR="true">
        			<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
        					pathname=""
        					memcachedNodes="n1:x.x.x.x:11211"
        					username="Username"
        					password="PW"
        					sticky="false"
        			        lockingMode="none"
        					sessionBackupAsync="true"
        					sessionBackupTimeout="10000"
        					operationTimeout="10000"
        					requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" 
        					transcoderFactoryClass="de.javakaffee.web.msm.JavaSerializationTranscoderFactory"/>
        		</Context>
        	  </Host>
            </Engine>
          </Service>
        </Server>
        

        My test scenario is following:

        • Startup both tomcats
        • Login into my application
        • Trigger one Function
        • STOP one Tomcat (this one, which I'm currently be)
        • Check if my Loadbalancer work fine (which works fine)
        • Try any request in my application
        • Exception in catalina-log from currently running tomcat:
        23-Jun-2016 16:51:43.474 FINE [ajp-apr-8009-exec-25] de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached Loading session from memcached: F3229A3A2BC720CD7A982F4E62E6493B-n1
        23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.JavaSerializationTranscoder.writeAttributes   storing attribute 'org.apache.myfaces.extensions.cdi.core.api.scope.conversation.WindowContext:EXISTING_WINDOW_ID_LIST' with value '[]'
        23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.BackupSessionTask.doBackupSession Trying to store session in memcached: 7A6A833C1F066B41851DFEC03491BD94-n1
        23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.BackupSessionTask.call Finished for session id 7A6A833C1F066B41851DFEC03491BD94-n1, returning status SUCCESS
        23-Jun-2016 16:51:43.490 WARNING [ajp-apr-8009-exec-25] de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached Could not load session with id F3229A3A2BC720CD7A982F4E62E6493B-n1 from memcached.
         java.lang.NullPointerException
        	at org.apache.webbeans.proxy.NormalScopeProxyFactory.createNormalScopeProxy(NormalScopeProxyFactory.java:121)
        	at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:125)
        	at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:497)
        	at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at java.util.HashMap.readObject(HashMap.java:1396)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:497)
        	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54)
        	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1842)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1799)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at de.javakaffee.web.msm.JavaSerializationTranscoder.deserializeAttributes(JavaSerializationTranscoder.java:172)
        	at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:171)
        	at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:128)
        	at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1100)
        	at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:585)
        	at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:201)
        	at org.apache.catalina.connector.Request.doGetSession(Request.java:2868)
        	at org.apache.catalina.connector.Request.getSession(Request.java:2260)
        	at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:895)
        	at org.apache.myfaces.context.servlet.SessionMap._getSession(SessionMap.java:87)
        	at org.apache.myfaces.context.servlet.SessionMap.getAttribute(SessionMap.java:49)
        	at org.apache.myfaces.util.AbstractThreadSafeAttributeMap.put(AbstractThreadSafeAttributeMap.java:108)
        	at org.apache.myfaces.util.AbstractThreadSafeAttributeMap.put(AbstractThreadSafeAttributeMap.java:38)
        	at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.cacheWindowId(ConversationUtils.java:232)
        	at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.resolveWindowContextId(ConversationUtils.java:163)
        	at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.tryToRestoreTheWindowIdEagerly(ConversationUtils.java:717)
        	at org.apache.myfaces.extensions.cdi.jsf2.impl.listener.phase.CodiLifecycleWrapper.broadcastBeforeFacesRequestEvent(CodiLifecycleWrapper.java:164)
        	at org.apache.myfaces.extensions.cdi.jsf2.impl.listener.phase.CodiLifecycleWrapper.execute(CodiLifecycleWrapper.java:81)
        	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:198)
        	at org.apache.myfaces.webapp.MyFacesServlet.service(MyFacesServlet.java:112)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        	at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:100)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        	at com.applicationName.appName.utils.web.servlet.filter.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
        	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
        	at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99)
        	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
        	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
        	at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:156)
        	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
        	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
        	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521)
        	at org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:850)
        	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674)
        	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2500)
        	at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2489)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        	at java.lang.Thread.run(Thread.java:745)
        

        I currently using OWB 1.6.2.

        My session database which I use for session replication is couchbase in there the sessionId's are definitely persisted.

        Show
        hurrrrricaaaaane Josef Puff added a comment - I've the maybe the same issue. In my case I have two different Windows Server 2008 R2 with JDK 1.8.0_XX. On both there is running a Tomcat 8.0.30 with the following server.xml configuration: <?xml version='1.0' encoding='utf-8'?> <Server port= "8005" shutdown= "SHUTDOWN" > <Listener className= "org.apache.catalina.startup.VersionLoggerListener" /> <Listener className= "org.apache.catalina.core.AprLifecycleListener" SSLEngine= "on" /> <Listener className= "org.apache.catalina.core.JreMemoryLeakPreventionListener" /> <Listener className= "org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" /> <Listener className= "org.apache.catalina.core.ThreadLocalLeakPreventionListener" /> <Service name= "Catalina" > <Connector port= "8080" protocol= "HTTP/1.1" connectionTimeout= "20000" redirectPort= "8443" /> <Connector port= "8009" enableLookups= "false" redirectPort= "8443" protocol= "AJP/1.3" backlog= "50" maxThreads= "500" minSpareThreads= "25" maxSpareThreads= "50" tcpNoDelay= "true" soTimeout= "60000" tomcatAuthentication= "false" /> <Engine name= "Catalina" defaultHost= "asmqa.tradeplace.com" > <Host name= "asmqa.tradeplace.com" appBase= "D:/WWW/applicationName/webapps" unpackWARs= "true" autoDeploy= "false" deployOnStartup= "false" > <Context path= "/tp4-web" docBase= "D:/WWW/applicationName/webapps/applicationName.war" reloadable= "false" unpackWAR= "true" > <Manager className= "de.javakaffee.web.msm.MemcachedBackupSessionManager" pathname="" memcachedNodes= "n1:x.x.x.x:11211" username= "Username" password= "PW" sticky= "false" lockingMode= "none" sessionBackupAsync= "true" sessionBackupTimeout= "10000" operationTimeout= "10000" requestUriIgnorePattern= ".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass= "de.javakaffee.web.msm.JavaSerializationTranscoderFactory" /> </Context> </Host> </Engine> </Service> </Server> My test scenario is following: Startup both tomcats Login into my application Trigger one Function STOP one Tomcat (this one, which I'm currently be) Check if my Loadbalancer work fine (which works fine) Try any request in my application Exception in catalina-log from currently running tomcat: 23-Jun-2016 16:51:43.474 FINE [ajp-apr-8009-exec-25] de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached Loading session from memcached: F3229A3A2BC720CD7A982F4E62E6493B-n1 23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.JavaSerializationTranscoder.writeAttributes storing attribute 'org.apache.myfaces.extensions.cdi.core.api.scope.conversation.WindowContext:EXISTING_WINDOW_ID_LIST' with value '[]' 23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.BackupSessionTask.doBackupSession Trying to store session in memcached: 7A6A833C1F066B41851DFEC03491BD94-n1 23-Jun-2016 16:51:43.474 FINE [msm-storage-thread-1] de.javakaffee.web.msm.BackupSessionTask.call Finished for session id 7A6A833C1F066B41851DFEC03491BD94-n1, returning status SUCCESS 23-Jun-2016 16:51:43.490 WARNING [ajp-apr-8009-exec-25] de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached Could not load session with id F3229A3A2BC720CD7A982F4E62E6493B-n1 from memcached. java.lang.NullPointerException at org.apache.webbeans.proxy.NormalScopeProxyFactory.createNormalScopeProxy(NormalScopeProxyFactory.java:121) at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:125) at sun.reflect.GeneratedMethodAccessor149.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1148) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at java.util.HashMap.readObject(HashMap.java:1396) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54) at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1842) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1799) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at de.javakaffee.web.msm.JavaSerializationTranscoder.deserializeAttributes(JavaSerializationTranscoder.java:172) at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:171) at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:128) at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1100) at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:585) at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:201) at org.apache.catalina.connector.Request.doGetSession(Request.java:2868) at org.apache.catalina.connector.Request.getSession(Request.java:2260) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:895) at org.apache.myfaces.context.servlet.SessionMap._getSession(SessionMap.java:87) at org.apache.myfaces.context.servlet.SessionMap.getAttribute(SessionMap.java:49) at org.apache.myfaces.util.AbstractThreadSafeAttributeMap.put(AbstractThreadSafeAttributeMap.java:108) at org.apache.myfaces.util.AbstractThreadSafeAttributeMap.put(AbstractThreadSafeAttributeMap.java:38) at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.cacheWindowId(ConversationUtils.java:232) at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.resolveWindowContextId(ConversationUtils.java:163) at org.apache.myfaces.extensions.cdi.jsf.impl.util.ConversationUtils.tryToRestoreTheWindowIdEagerly(ConversationUtils.java:717) at org.apache.myfaces.extensions.cdi.jsf2.impl.listener.phase.CodiLifecycleWrapper.broadcastBeforeFacesRequestEvent(CodiLifecycleWrapper.java:164) at org.apache.myfaces.extensions.cdi.jsf2.impl.listener.phase.CodiLifecycleWrapper.execute(CodiLifecycleWrapper.java:81) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:198) at org.apache.myfaces.webapp.MyFacesServlet.service(MyFacesServlet.java:112) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:100) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.applicationName.appName.utils.web.servlet.filter.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:71) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:156) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:521) at org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:850) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:674) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2500) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2489) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) I currently using OWB 1.6.2. My session database which I use for session replication is couchbase in there the sessionId's are definitely persisted.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Maybe ensure to clean you conf/Catalina and work/ dirs and restart. If some old code ref leaked then you will get that.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Maybe ensure to clean you conf/Catalina and work/ dirs and restart. If some old code ref leaked then you will get that.
        Hide
        struberg Mark Struberg added a comment -

        Let's be more precise:

        • It should work perfectly fine if you shut down the app and then restart it. If this is broken, then we need to investigate.
        • It is expected to blow up if you persist the session, then e.g. add a Qualifier or change the class name and restart. This will lead to a different passivationId. Of course we should blow up with a more meaningful message than we do now
        Show
        struberg Mark Struberg added a comment - Let's be more precise: It should work perfectly fine if you shut down the app and then restart it. If this is broken, then we need to investigate. It is expected to blow up if you persist the session, then e.g. add a Qualifier or change the class name and restart. This will lead to a different passivationId. Of course we should blow up with a more meaningful message than we do now
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Yes that's it, you reuse an old bean. Just disable session persistence then.

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Yes that's it, you reuse an old bean. Just disable session persistence then.
        Hide
        balusc Bauke Scholtz added a comment -

        Here's the one from TomEE 7.0.0, freshly downloaded and unzipped today:

        Jun 10, 2016 12:12:24 PM org.apache.catalina.session.StandardManager startInternal
        SEVERE: Exception loading sessions from persistent storage
        java.lang.NullPointerException
        	at org.apache.webbeans.proxy.NormalScopeProxyFactory.createNormalScopeProxy(NormalScopeProxyFactory.java:121)
        	at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:125)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:497)
        	at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at java.util.HashMap.readObject(HashMap.java:1396)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:497)
        	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
        	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54)
        	at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1840)
        	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1799)
        	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
        	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
        	at org.apache.catalina.session.StandardSession.doReadObject(StandardSession.java:1611)
        	at org.apache.catalina.session.StandardSession.readObjectData(StandardSession.java:1077)
        	at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:218)
        	at org.apache.catalina.session.StandardManager.load(StandardManager.java:162)
        	at org.apache.catalina.session.StandardManager.startInternal(StandardManager.java:356)
        	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:152)
        	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5212)
        	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:152)
        	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1405)
        	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1395)
        	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at java.lang.Thread.run(Thread.java:745)
        
        Show
        balusc Bauke Scholtz added a comment - Here's the one from TomEE 7.0.0, freshly downloaded and unzipped today: Jun 10, 2016 12:12:24 PM org.apache.catalina.session.StandardManager startInternal SEVERE: Exception loading sessions from persistent storage java.lang.NullPointerException at org.apache.webbeans.proxy.NormalScopeProxyFactory.createNormalScopeProxy(NormalScopeProxyFactory.java:121) at org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.readResolve(NormalScopedBeanInterceptorHandler.java:125) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1104) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1810) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at java.util.HashMap.readObject(HashMap.java:1396) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at org.apache.webbeans.context.PassivatingContext.readExternal(PassivatingContext.java:54) at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1840) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1799) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371) at org.apache.catalina.session.StandardSession.doReadObject(StandardSession.java:1611) at org.apache.catalina.session.StandardSession.readObjectData(StandardSession.java:1077) at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:218) at org.apache.catalina.session.StandardManager.load(StandardManager.java:162) at org.apache.catalina.session.StandardManager.startInternal(StandardManager.java:356) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:152) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5212) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:152) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1405) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1395) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang. Thread .run( Thread .java:745)
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Ah, can you share the full stacktrace then?

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Ah, can you share the full stacktrace then?
        Hide
        balusc Bauke Scholtz added a comment - - edited

        No, it was a single server which was just restarted without changes in code/deployments. If that's relevant, I'm managing it via Eclipse.

        Show
        balusc Bauke Scholtz added a comment - - edited No, it was a single server which was just restarted without changes in code/deployments. If that's relevant, I'm managing it via Eclipse.
        Hide
        romain.manni-bucau Romain Manni-Bucau added a comment -

        Bauke Scholtz means the bean was not found from its beanPassivationId. Can it be you have 2 different application versions in a cluster? or redelpoyed a new version and kept the old session referencing the old instance?

        Show
        romain.manni-bucau Romain Manni-Bucau added a comment - Bauke Scholtz means the bean was not found from its beanPassivationId. Can it be you have 2 different application versions in a cluster? or redelpoyed a new version and kept the old session referencing the old instance?
        Hide
        balusc Bauke Scholtz added a comment -

        FYI: issue is still reproducible in OWB 1.6.3 (as present in TomEE 7.0.0).

        Show
        balusc Bauke Scholtz added a comment - FYI: issue is still reproducible in OWB 1.6.3 (as present in TomEE 7.0.0).
        Hide
        struberg Mark Struberg added a comment -

        Txs for the report, Bauke, will check this soon. Just need to prepare talks for next weeks conference. After that I'll dig into it.

        Show
        struberg Mark Struberg added a comment - Txs for the report, Bauke, will check this soon. Just need to prepare talks for next weeks conference. After that I'll dig into it.

          People

          • Assignee:
            struberg Mark Struberg
            Reporter:
            balusc Bauke Scholtz
          • Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development