Uploaded image for project: 'Log4j 2'
  1. Log4j 2
  2. LOG4J2-1934

JMS Appender does not know how to recover from a broken connection

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8.2
    • Fix Version/s: 2.9.0
    • Component/s: Appenders
    • Labels:
      None

      Description

      If a broker, like ActiveMQ goes up, down, and up. Log4j gives me:

      2017-06-06 18:26:58,396 HPM-ListeningIoReactor 4 ERROR An exception occurred processing Appender ActiveMQ org.apache.logging.log4j.core.appender.AppenderLoggingException: javax.jms.IllegalStateException: The Session is closed
      	at org.apache.logging.log4j.core.appender.mom.JmsAppender.append(JmsAppender.java:69)
      	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
      	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
      	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
      	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
      	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448)
      	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433)
      	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
      	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403)
      	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
      	at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
      	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2091)
      	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1813)
      	at org.apache.logging.log4j.spi.AbstractLogger.info(AbstractLogger.java:1267)
      	>SNIP<
      	at org.apache.http.nio.protocol.HttpAsyncService.requestReceived(HttpAsyncService.java:338)
      	at org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(DefaultNHttpServerConnection.java:273)
      	at org.apache.http.impl.nio.DefaultHttpServerIODispatch.onInputReady(DefaultHttpServerIODispatch.java:158)
      	at org.apache.http.impl.nio.DefaultHttpServerIODispatch.onInputReady(DefaultHttpServerIODispatch.java:52)
      	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
      	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
      	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
      	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
      	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
      	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
      	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
      	at java.lang.Thread.run(Unknown Source)
      Caused by: javax.jms.IllegalStateException: The Session is closed
      	at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:771)
      	at org.apache.activemq.ActiveMQSession.configureMessage(ActiveMQSession.java:759)
      	at org.apache.activemq.ActiveMQSession.createMapMessage(ActiveMQSession.java:338)
      	at org.apache.logging.log4j.core.appender.mom.JmsManager.createMessage(JmsManager.java:136)
      	at org.apache.logging.log4j.core.appender.mom.JmsAppender.append(JmsAppender.java:65)
      	... 28 more
      

      I plan on fixing that ASAP.

        Activity

        Hide
        jvz Matt Sicker added a comment -

        As a workaround, can't you use a failover broker URL when connecting to ActiveMQ to allow the driver to auto-reconnect?

        Show
        jvz Matt Sicker added a comment - As a workaround, can't you use a failover broker URL when connecting to ActiveMQ to allow the driver to auto-reconnect?
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit c9ae6a85f46c5083f5b63c3442afd4e778291429 in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=c9ae6a8 ]

        LOG4J2-1934
        JMS Appender does not know how to recover from a broken connection.
        Refactor JMS Appender tests for Apache ActiveMQ and add new tests that
        demonstrate the issue. Failing tests are annotated with @Ignore.

        Show
        jira-bot ASF subversion and git services added a comment - Commit c9ae6a85f46c5083f5b63c3442afd4e778291429 in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=c9ae6a8 ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection. Refactor JMS Appender tests for Apache ActiveMQ and add new tests that demonstrate the issue. Failing tests are annotated with @Ignore.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 8573c8e563c2b734f17e6c7f4a141b7cfb80a286 in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=8573c8e ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 8573c8e563c2b734f17e6c7f4a141b7cfb80a286 in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=8573c8e ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection.
        Hide
        garydgregory Gary Gregory added a comment -

        I am looking for a code review from the community. I've written new tests to match the main reconnect aspect of this feature but more eyes on this would be nice.

        Show
        garydgregory Gary Gregory added a comment - I am looking for a code review from the community. I've written new tests to match the main reconnect aspect of this feature but more eyes on this would be nice.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 19b20c0e75f71de5d5d04f6e1a4878249e88c81d in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=19b20c0 ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection. Remove incorrect comment.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 19b20c0e75f71de5d5d04f6e1a4878249e88c81d in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=19b20c0 ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection. Remove incorrect comment.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5d1b605d922d37c02afdd656ccdb4afae2864cd0 in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=5d1b605 ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5d1b605d922d37c02afdd656ccdb4afae2864cd0 in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=5d1b605 ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5d1b605d922d37c02afdd656ccdb4afae2864cd0 in logging-log4j2's branch refs/heads/LOG4J2-1958 from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=5d1b605 ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5d1b605d922d37c02afdd656ccdb4afae2864cd0 in logging-log4j2's branch refs/heads/ LOG4J2-1958 from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=5d1b605 ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection.
        Hide
        ralph.goers@dslextreme.com Ralph Goers added a comment - - edited

        I just took a look at the commit and I do NOT like this change. As I expected from your description you are creating new managers in the appender to reconnect. The responsibility for reconnecting should reside inside the manager, not in the Appender. Please redo it.

        Show
        ralph.goers@dslextreme.com Ralph Goers added a comment - - edited I just took a look at the commit and I do NOT like this change. As I expected from your description you are creating new managers in the appender to reconnect. The responsibility for reconnecting should reside inside the manager, not in the Appender. Please redo it.
        Hide
        garydgregory Gary Gregory added a comment -

        Hi Ralph,

        Thank you for the review.

        My motivation for the current solution was to implement this in a somewhat symmetrical way to reconfigure:

        Reconfigure: throw away the Appender and reuse the cached Manager.

        Recover: throw away the Manager and reuse the Appender.

        Otherwise, I would just implement a kind of manager inside the Manager...

        Further thoughts?

        Gary

        Show
        garydgregory Gary Gregory added a comment - Hi Ralph, Thank you for the review. My motivation for the current solution was to implement this in a somewhat symmetrical way to reconfigure: Reconfigure: throw away the Appender and reuse the cached Manager. Recover: throw away the Manager and reuse the Appender. Otherwise, I would just implement a kind of manager inside the Manager... Further thoughts? Gary
        Hide
        ralph.goers@dslextreme.com Ralph Goers added a comment -

        As I suggested before, take a look at how the TCP Socket Manager handles reconnection.

        Show
        ralph.goers@dslextreme.com Ralph Goers added a comment - As I suggested before, take a look at how the TCP Socket Manager handles reconnection.
        Hide
        garydgregory Gary Gregory added a comment -

        I did and I will rewrite the JMS appender.

        Show
        garydgregory Gary Gregory added a comment - I did and I will rewrite the JMS appender.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit de7487ed22b03f5c1a73ddb8f8d0f8bc89b8f4d8 in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=de7487e ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection.
        LOG4J2-1955JMS Appender should be able connect to a broker (later)
        even it is not present at configuration time.

        Show
        jira-bot ASF subversion and git services added a comment - Commit de7487ed22b03f5c1a73ddb8f8d0f8bc89b8f4d8 in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=de7487e ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection. LOG4J2-1955 JMS Appender should be able connect to a broker (later) even it is not present at configuration time.
        Hide
        garydgregory Gary Gregory added a comment -

        In Git master: Recoded to follow the Reconnect thread pattern from TcpSocketManager. Looking for a peer-review.

        Show
        garydgregory Gary Gregory added a comment - In Git master: Recoded to follow the Reconnect thread pattern from TcpSocketManager . Looking for a peer-review.
        Hide
        mikaelstaldal Mikael Ståldal added a comment -

        The documentation should mention the default values for the two new properties on JmsAppender.

        Show
        mikaelstaldal Mikael Ståldal added a comment - The documentation should mention the default values for the two new properties on JmsAppender.
        Hide
        mikaelstaldal Mikael Ståldal added a comment -

        I think the comment on JmsManager.Reconnector is wrong (copy-paste).

        Show
        mikaelstaldal Mikael Ståldal added a comment - I think the comment on JmsManager.Reconnector is wrong (copy-paste).
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 55f80f35060d60788393714514b1c27aa5773095 in logging-log4j2's branch refs/heads/master from Gary Gregory
        [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=55f80f3 ]

        LOG4J2-1934 JMS Appender does not know how to recover from a broken
        connection.
        LOG4J2-1955JMS Appender should be able connect to a broker (later)
        even it is not present at configuration time.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 55f80f35060d60788393714514b1c27aa5773095 in logging-log4j2's branch refs/heads/master from Gary Gregory [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=55f80f3 ] LOG4J2-1934 JMS Appender does not know how to recover from a broken connection. LOG4J2-1955 JMS Appender should be able connect to a broker (later) even it is not present at configuration time.
        Hide
        garydgregory Gary Gregory added a comment - - edited

        I added a "Default" column to the table for the JMA Appender. Thank you for noticing the missing information Mikael Ståldal!

        Show
        garydgregory Gary Gregory added a comment - - edited I added a "Default" column to the table for the JMA Appender. Thank you for noticing the missing information Mikael Ståldal !

          People

          • Assignee:
            garydgregory Gary Gregory
            Reporter:
            garydgregory Gary Gregory
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development