Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0
    • Component/s: Appenders
    • Labels:
      None

      Description

      Inspired by this mailing list post. This would in one swoop support various APIs such as:

      • Slack and other ChatOps software
      • ElasticSearch
      • ...I'm sure there's more

      Basic features that would be needed are:

      • HTTP verb
      • headers
      • body

        Issue Links

          Activity

          Hide
          jvz Matt Sicker added a comment -

          After using Elasticsearch at work lately, I've found that having the ability to directly append (using the bulk add API) log events to ES would be nicer than having to add Logstash into the mix. Some work may have to be done at enhancing the JsonLayout or some way to specify custom JSON layouts, but I've seen that feature request elsewhere as well.

          Show
          jvz Matt Sicker added a comment - After using Elasticsearch at work lately, I've found that having the ability to directly append (using the bulk add API) log events to ES would be nicer than having to add Logstash into the mix. Some work may have to be done at enhancing the JsonLayout or some way to specify custom JSON layouts, but I've seen that feature request elsewhere as well.
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          Since it is quite common to use Elasticsearch for log aggregation, and it quite annoying to have to use Logstach, would it make sense with a specific Elasticsearch appender?

          Show
          mikaelstaldal Mikael Ståldal added a comment - Since it is quite common to use Elasticsearch for log aggregation, and it quite annoying to have to use Logstach, would it make sense with a specific Elasticsearch appender?
          Hide
          jvz Matt Sicker added a comment -

          I think so. It could extend an http appender similar to how we abstract out the OutputStream appenders. The ES API is pretty straightforward REST, so it wouldn't be too hard.

          Show
          jvz Matt Sicker added a comment - I think so. It could extend an http appender similar to how we abstract out the OutputStream appenders. The ES API is pretty straightforward REST, so it wouldn't be too hard.
          Hide
          jvz Matt Sicker added a comment -

          I was thinking about this the other night. It may be useful to abstract the HttpClient idea here similar to what Spring does so that an HttpClient implementation can be provided at runtime. The default one would just use HttpURLConnection, but an Apache HttpClient and Netty version would be useful, too. Of course, this should be a minimal HttpClient adapter and not some full featured client like the Apache one.

          Show
          jvz Matt Sicker added a comment - I was thinking about this the other night. It may be useful to abstract the HttpClient idea here similar to what Spring does so that an HttpClient implementation can be provided at runtime. The default one would just use HttpURLConnection, but an Apache HttpClient and Netty version would be useful, too. Of course, this should be a minimal HttpClient adapter and not some full featured client like the Apache one.
          Hide
          jvz Matt Sicker added a comment -

          With the newly added JSON escape pattern converter, arbitrary JSON messages can be sent via this plugin which should provide enough functionality to create whatever payload is necessary to use a webhook or basic REST API.

          Show
          jvz Matt Sicker added a comment - With the newly added JSON escape pattern converter, arbitrary JSON messages can be sent via this plugin which should provide enough functionality to create whatever payload is necessary to use a webhook or basic REST API.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 410f9d360eabcdc6949c75973b786c9e2cec9c66 in logging-log4j2's branch refs/heads/LOG4J2-1442 from Mikael Ståldal
          [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=410f9d3 ]

          LOG4J2-1442 Generic HTTP appender

          Show
          jira-bot ASF subversion and git services added a comment - Commit 410f9d360eabcdc6949c75973b786c9e2cec9c66 in logging-log4j2's branch refs/heads/ LOG4J2-1442 from Mikael Ståldal [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=410f9d3 ] LOG4J2-1442 Generic HTTP appender
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 0af515f3b1ec67422de254d08ac87b135e3c6923 in logging-log4j2's branch refs/heads/LOG4J2-1442 from Mikael Ståldal
          [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=0af515f ]

          Merge branch 'master' into LOG4J2-1442

          Show
          jira-bot ASF subversion and git services added a comment - Commit 0af515f3b1ec67422de254d08ac87b135e3c6923 in logging-log4j2's branch refs/heads/ LOG4J2-1442 from Mikael Ståldal [ https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=0af515f ] Merge branch 'master' into LOG4J2-1442
          Hide
          mikaelstaldal Mikael Ståldal added a comment - - edited

          I have this almost finished now, have a look at Git branch LOG4J2-1442. Matt Sicker

          Show
          mikaelstaldal Mikael Ståldal added a comment - - edited I have this almost finished now, have a look at Git branch LOG4J2-1442 . Matt Sicker
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          In master branch. Please verify and close.

          Show
          mikaelstaldal Mikael Ståldal added a comment - In master branch. Please verify and close.
          Hide
          jvz Matt Sicker added a comment -

          Looks pretty good! You can inject a URL directly in the builder class, though. No need to convert in the manager. If you want, you could also use URI instead for portability, though both classes seem pretty portable (though using URI would allow for some rather arbitrary URIs that aren't really HTTP-related, so URL does make sense).

          Show
          jvz Matt Sicker added a comment - Looks pretty good! You can inject a URL directly in the builder class, though. No need to convert in the manager. If you want, you could also use URI instead for portability, though both classes seem pretty portable (though using URI would allow for some rather arbitrary URIs that aren't really HTTP-related, so URL does make sense).
          Hide
          jvz Matt Sicker added a comment -

          Some other code review comments:

          Show
          jvz Matt Sicker added a comment - Some other code review comments: Can use try-with-resources here: https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/HttpURLConnectionManager.java#L109 This connection should probably be cached. This allows for the connection to be reused which seems like a useful scenario for higher frequency logging. A feature request here may be to support batched log messages, but that would require more advanced configuration most likely.
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          Good idea, using java.net.URL in builder now.

          Show
          mikaelstaldal Mikael Ståldal added a comment - Good idea, using java.net.URL in builder now.
          Hide
          garydgregory Gary Gregory added a comment - - edited

          As soon as I read "you should cache the connection..." I got the feeling that this will only be the first of many tweaks toward writing an HTTP client when we already have a great one over in the Apache HttpComponents project hc.apache.org

          Show
          garydgregory Gary Gregory added a comment - - edited As soon as I read "you should cache the connection..." I got the feeling that this will only be the first of many tweaks toward writing an HTTP client when we already have a great one over in the Apache HttpComponents project hc.apache.org
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          Yes, using more try-with-resources now.

          Show
          mikaelstaldal Mikael Ståldal added a comment - Yes, using more try-with-resources now.
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          Is it really possible to cache the connection? Javadocs says:

          Each HttpURLConnection instance is used to make a single request

          Show
          mikaelstaldal Mikael Ståldal added a comment - Is it really possible to cache the connection? Javadocs says: Each HttpURLConnection instance is used to make a single request
          Hide
          mikaelstaldal Mikael Ståldal added a comment -

          As Gary Gregory suggests, if we want something more fancy, we should probably use Apache HttpComponents.

          Show
          mikaelstaldal Mikael Ståldal added a comment - As Gary Gregory suggests, if we want something more fancy, we should probably use Apache HttpComponents.
          Hide
          jvz Matt Sicker added a comment -

          Oh wait, I read more in the docs, you don't need to cache the connection object. It has a pool beneath the covers or something similar.

          Show
          jvz Matt Sicker added a comment - Oh wait, I read more in the docs, you don't need to cache the connection object. It has a pool beneath the covers or something similar.

            People

            • Assignee:
              mikaelstaldal Mikael Ståldal
              Reporter:
              jvz Matt Sicker
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development