Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Done
-
master
-
None
Description
== Context
As any application, James outputs logs.
The team uses the SLF4J logging facade, along with the logback-classic log producer library.
The team had been setting up contextual logging: the context of the execution is recorded through the use of the
MDC (Mapped Diagnostic Context) on the behalf of the developper. Once the context set up, all logging action performed in the scope will implicilty include the MDC context.
Contextual logging is beneficial as it enable dispaying efficiently request parameters, tracking users, request ids and so on. It greatly enhence the debugging experience.
Historically, we relyed on a custom logback appender to output logs directly to ElasticSearch, in a structured fashion, that could be directly accessed via Kibana.
The historical stack had been proven brittle:
- James needs to be pushing logs, eventual consummiung some of its resources
- Being push based, upon bolks or temporary ElasticSearch failures, some logs are lost.
This conversation was marked as resolved by chibenwa - Logs are duplicated, both to the standard output and ElasticSearch
- Changes to the logging stack requires a James restart, that could cause inconsistency and harm availability.
This conversation was marked as resolved by chibenwa
== Desirable propoerties of the logging stack
We see the following properties as desirable:
- Having structured logging
- Being pull based
- Uncorrelating the logging stack from the James configuration
== Definition of Done
Document how to rely on FluentD to collect James logs. The James logging configuration can be simplified to log all logs to the standard output.
Document how to emmit logs in a JSON format, in order for fluentD and Kibana to allow preserving the structured logging experience.
== Consequences
We expect our logging stack to be more resilient, while keeping structured logging.
https://github.com/linagora/logback-elasticsearch-appender project can be deprecated - if desired.