Log4j 2
  1. Log4j 2
  2. LOG4J2-623

Better structure of Thread Context Map in JSONLayout

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0-rc1, 2.0, 2.0.1
    • Fix Version/s: 2.0-rc2
    • Component/s: Layouts
    • Labels:
      None

      Description

      Currently, the Thread Context Map looks like this in JSONLayout:

      Current
       "Properties":[
            {
              "name":"UserName",
              "value":"admin"
            },
            {
              "name":"OrgName",
              "value":"test"
            } 
          ] 
      

      This does not properly make use of the JSON data format. Since the Thread Context Map is a map, it should be represented as a JSON object. And why not name it "mdc" rather than the quite vauge "Properties"?

      Suggested
        "mdc": {
          "UserName":"admin", 
          "OrgName":"test"
        }
      

        Issue Links

          Activity

          Hide
          Gary Gregory added a comment -

          I am reconsidering some details of the JSON layout as I work through the JSON and XML receivers. This is WIP and I have been busy with other more urgent matters. I hope to return to it soon.

          Gary

          Show
          Gary Gregory added a comment - I am reconsidering some details of the JSON layout as I work through the JSON and XML receivers. This is WIP and I have been busy with other more urgent matters. I hope to return to it soon. Gary
          Hide
          Ralph Goers added a comment -

          I would expect that the JSONLayout would mimic the XMLLayout to some degree.

          Show
          Ralph Goers added a comment - I would expect that the JSONLayout would mimic the XMLLayout to some degree.
          Hide
          Mikael Ståldal added a comment -

          Currently, it does mimic the XMLLayout. This is not a good thing in this case since JSON and XML have different structures.

          Show
          Mikael Ståldal added a comment - Currently, it does mimic the XMLLayout. This is not a good thing in this case since JSON and XML have different structures.
          Hide
          Ralph Goers added a comment -

          I didn't mean in terms of structure. I mean the names in JSON and XML should match where it makes sense.

          Show
          Ralph Goers added a comment - I didn't mean in terms of structure. I mean the names in JSON and XML should match where it makes sense.
          Hide
          Mikael Ståldal added a comment -

          Ah, yes then I agree.

          Show
          Mikael Ståldal added a comment - Ah, yes then I agree.
          Hide
          Ralph Goers added a comment -

          This has been fixed in revision 1593727. I left the tag name as "Properties" since that is what the XMLLayout uses, but it now generates a proper JSON map. Please verify and close.

          Show
          Ralph Goers added a comment - This has been fixed in revision 1593727. I left the tag name as "Properties" since that is what the XMLLayout uses, but it now generates a proper JSON map. Please verify and close.
          Hide
          Mikael Ståldal added a comment -

          The syntax of the generated JSON is invalid, it looks like this:

            { 
              "logger":"MyLogger",
              "timestamp":"1399900017525",
              "level":"INFO",
              "thread":"qtp18151360-47",
              "message":"Doing stuff...",
              "Properties": {
                "RequesterIp": "127.0.0.1"},
                "UserName": "admin"},
                "OrgName": "test"}
              }
            }
          

          There are superfluous '}' after each property.

          Show
          Mikael Ståldal added a comment - The syntax of the generated JSON is invalid, it looks like this: { "logger" : "MyLogger" , "timestamp" : "1399900017525" , "level" : "INFO" , "thread" : "qtp18151360-47" , "message" : "Doing stuff..." , "Properties" : { "RequesterIp" : "127.0.0.1" }, "UserName" : "admin" }, "OrgName" : "test" } } } There are superfluous '}' after each property.
          Hide
          Gary Gregory added a comment -

          I am putting the finishing touches on redoing the XML and JSON layouts and receivers based on Jackson. Maps in XML and JSON will not be "somekey":"somevalue" because you cannot write a schema that will works for all JSON and XML documents with this kind of document.

          The XML will be like:

          	<ContextMap>
          		<item key="MDC.B" value="B_Value"/>
          		<item key="MDC.A" value="A_Value"/>
          	</ContextMap>
          

          and the JSON will be equivalent.

          Show
          Gary Gregory added a comment - I am putting the finishing touches on redoing the XML and JSON layouts and receivers based on Jackson. Maps in XML and JSON will not be "somekey":"somevalue" because you cannot write a schema that will works for all JSON and XML documents with this kind of document. The XML will be like: <ContextMap> <item key= "MDC.B" value= "B_Value" /> <item key= "MDC.A" value= "A_Value" /> </ContextMap> and the JSON will be equivalent.
          Hide
          Mikael Ståldal added a comment -

          Can you give a concrete example of what the JSON will look like?

          It is not sufficient to say that it will be "equivalent" to the XML format, since JSON and XML are structurally different.

          Show
          Mikael Ståldal added a comment - Can you give a concrete example of what the JSON will look like? It is not sufficient to say that it will be "equivalent" to the XML format, since JSON and XML are structurally different.
          Hide
          Ralph Goers added a comment -

          Gary, the correct syntax for a map in JSON is

          "mapName: {
           "MDC.B": "B.Value",
           "MDC.A": "A.Value"
          }
          

          not the way it was previously generated. The Jackson docs specifically mention that you can serialize and deserialize a Map with this syntax so I don't know why you would be having a problem.

          Show
          Ralph Goers added a comment - Gary, the correct syntax for a map in JSON is "mapName: { "MDC.B" : "B.Value" , "MDC.A" : "A.Value" } not the way it was previously generated. The Jackson docs specifically mention that you can serialize and deserialize a Map with this syntax so I don't know why you would be having a problem.
          Hide
          Ralph Goers added a comment -

          I have removed the incorrect trailing '} in revision 1593981. Please verify and close.

          Show
          Ralph Goers added a comment - I have removed the incorrect trailing '} in revision 1593981. Please verify and close.
          Hide
          Mikael Ståldal added a comment -

          Now I get this:

            {
              "logger":"MyLogger",
              "timestamp":"1399907141921",
              "level":"INFO",
              "thread":"qtp18151360-47",
              "message":"Doing stuff...",
              "Properties": {
                "RequesterIp": "127.0.0.1",
                "UserName": "admin",
                "OrgName": "test"
              }
            }
          

          Which is correct and expected. Closing this issue.

          Show
          Mikael Ståldal added a comment - Now I get this: { "logger" : "MyLogger" , "timestamp" : "1399907141921" , "level" : "INFO" , "thread" : "qtp18151360-47" , "message" : "Doing stuff..." , "Properties" : { "RequesterIp" : "127.0.0.1" , "UserName" : "admin" , "OrgName" : "test" } } Which is correct and expected. Closing this issue.
          Hide
          Gary Gregory added a comment -

          You should expect the following before 2.0 (ASAP):

          {
            "timeMillis": 1,
            "thread": "MyThreadName",
            "level": "DEBUG",
            "loggerName": "a.B",
            "marker": {
              "name": "Marker1",
              "parents": [{
                "name": "ParentMarker1"
              }, {
                "name": "ParentMarker2"
              }]
            },
            "message": "Msg",
            "thrown": {
              "cause": {
                "name": "java.lang.NullPointerException",
                "message": "testNPEx",
                "localizedMessage": "testNPEx",
                "extendedStackTrace": [{
                  "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest",
                  "method": "testAllFeatures",
                  "file": "JSONLayoutTest.java",
                  "line": 122,
                  "exact": false,
                  "location": "test-classes/",
                  "version": "?"
                }, {
                  "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest",
                  "method": "testLocationOnCompactOnMdcOn",
                  "file": "JSONLayoutTest.java",
                  "line": 298,
                  "exact": false
                }, {
                  "class": "sun.reflect.NativeMethodAccessorImpl",
                  "method": "invoke0",
                  "line": -2,
                  "exact": false
                }, {
                  "class": "sun.reflect.NativeMethodAccessorImpl",
                  "method": "invoke",
                  "line": -1,
                  "exact": false
                }, {
                  "class": "sun.reflect.DelegatingMethodAccessorImpl",
                  "method": "invoke",
                  "line": -1,
                  "exact": false
                }, {
                  "class": "java.lang.reflect.Method",
                  "method": "invoke",
                  "line": -1,
                  "exact": false
                }, {
                  "class": "org.junit.runners.model.FrameworkMethod$1",
                  "method": "runReflectiveCall",
                  "file": "FrameworkMethod.java",
                  "line": 47,
                  "exact": false
                }, {
                  "class": "org.junit.internal.runners.model.ReflectiveCallable",
                  "method": "run",
                  "file": "ReflectiveCallable.java",
                  "line": 12,
                  "exact": false
                }, {
                  "class": "org.junit.runners.model.FrameworkMethod",
                  "method": "invokeExplosively",
                  "file": "FrameworkMethod.java",
                  "line": 44,
                  "exact": false
                }, {
                  "class": "org.junit.internal.runners.statements.InvokeMethod",
                  "method": "evaluate",
                  "file": "InvokeMethod.java",
                  "line": 17,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner",
                  "method": "runLeaf",
                  "file": "ParentRunner.java",
                  "line": 271,
                  "exact": false
                }, {
                  "class": "org.junit.runners.BlockJUnit4ClassRunner",
                  "method": "runChild",
                  "file": "BlockJUnit4ClassRunner.java",
                  "line": 70,
                  "exact": false
                }, {
                  "class": "org.junit.runners.BlockJUnit4ClassRunner",
                  "method": "runChild",
                  "file": "BlockJUnit4ClassRunner.java",
                  "line": 50,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner$3",
                  "method": "run",
                  "file": "ParentRunner.java",
                  "line": 238,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner$1",
                  "method": "schedule",
                  "file": "ParentRunner.java",
                  "line": 63,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner",
                  "method": "runChildren",
                  "file": "ParentRunner.java",
                  "line": 236,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner",
                  "method": "access$000",
                  "file": "ParentRunner.java",
                  "line": 53,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner$2",
                  "method": "evaluate",
                  "file": "ParentRunner.java",
                  "line": 229,
                  "exact": false
                }, {
                  "class": "org.junit.internal.runners.statements.RunBefores",
                  "method": "evaluate",
                  "file": "RunBefores.java",
                  "line": 26,
                  "exact": false
                }, {
                  "class": "org.junit.internal.runners.statements.RunAfters",
                  "method": "evaluate",
                  "file": "RunAfters.java",
                  "line": 27,
                  "exact": false
                }, {
                  "class": "org.junit.runners.ParentRunner",
                  "method": "run",
                  "file": "ParentRunner.java",
                  "line": 309,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference",
                  "method": "run",
                  "file": "JUnit4TestReference.java",
                  "line": 50,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit.runner.TestExecution",
                  "method": "run",
                  "file": "TestExecution.java",
                  "line": 38,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                  "method": "runTests",
                  "file": "RemoteTestRunner.java",
                  "line": 467,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                  "method": "runTests",
                  "file": "RemoteTestRunner.java",
                  "line": 683,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                  "method": "run",
                  "file": "RemoteTestRunner.java",
                  "line": 390,
                  "exact": false
                }, {
                  "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                  "method": "main",
                  "file": "RemoteTestRunner.java",
                  "line": 197,
                  "exact": false
                }],
                "commonElementCount": 26
              },
              "name": "java.io.IOException",
              "message": "testIOEx",
              "localizedMessage": "testIOEx",
              "extendedStackTrace": [{
                "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest",
                "method": "testAllFeatures",
                "file": "JSONLayoutTest.java",
                "line": 122,
                "exact": true,
                "location": "test-classes/",
                "version": "?"
              }, {
                "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest",
                "method": "testLocationOnCompactOnMdcOn",
                "file": "JSONLayoutTest.java",
                "line": 298,
                "exact": true,
                "location": "test-classes/",
                "version": "?"
              }, {
                "class": "sun.reflect.NativeMethodAccessorImpl",
                "method": "invoke0",
                "line": -2,
                "exact": false,
                "location": "?",
                "version": "1.7.0_55"
              }, {
                "class": "sun.reflect.NativeMethodAccessorImpl",
                "method": "invoke",
                "line": -1,
                "exact": false,
                "location": "?",
                "version": "1.7.0_55"
              }, {
                "class": "sun.reflect.DelegatingMethodAccessorImpl",
                "method": "invoke",
                "line": -1,
                "exact": false,
                "location": "?",
                "version": "1.7.0_55"
              }, {
                "class": "java.lang.reflect.Method",
                "method": "invoke",
                "line": -1,
                "exact": false,
                "location": "?",
                "version": "1.7.0_55"
              }, {
                "class": "org.junit.runners.model.FrameworkMethod$1",
                "method": "runReflectiveCall",
                "file": "FrameworkMethod.java",
                "line": 47,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.internal.runners.model.ReflectiveCallable",
                "method": "run",
                "file": "ReflectiveCallable.java",
                "line": 12,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.model.FrameworkMethod",
                "method": "invokeExplosively",
                "file": "FrameworkMethod.java",
                "line": 44,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.internal.runners.statements.InvokeMethod",
                "method": "evaluate",
                "file": "InvokeMethod.java",
                "line": 17,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner",
                "method": "runLeaf",
                "file": "ParentRunner.java",
                "line": 271,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.BlockJUnit4ClassRunner",
                "method": "runChild",
                "file": "BlockJUnit4ClassRunner.java",
                "line": 70,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.BlockJUnit4ClassRunner",
                "method": "runChild",
                "file": "BlockJUnit4ClassRunner.java",
                "line": 50,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner$3",
                "method": "run",
                "file": "ParentRunner.java",
                "line": 238,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner$1",
                "method": "schedule",
                "file": "ParentRunner.java",
                "line": 63,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner",
                "method": "runChildren",
                "file": "ParentRunner.java",
                "line": 236,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner",
                "method": "access$000",
                "file": "ParentRunner.java",
                "line": 53,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner$2",
                "method": "evaluate",
                "file": "ParentRunner.java",
                "line": 229,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.internal.runners.statements.RunBefores",
                "method": "evaluate",
                "file": "RunBefores.java",
                "line": 26,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.internal.runners.statements.RunAfters",
                "method": "evaluate",
                "file": "RunAfters.java",
                "line": 27,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.junit.runners.ParentRunner",
                "method": "run",
                "file": "ParentRunner.java",
                "line": 309,
                "exact": true,
                "location": "junit-4.11.jar",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference",
                "method": "run",
                "file": "JUnit4TestReference.java",
                "line": 50,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit.runner.TestExecution",
                "method": "run",
                "file": "TestExecution.java",
                "line": 38,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                "method": "runTests",
                "file": "RemoteTestRunner.java",
                "line": 467,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                "method": "runTests",
                "file": "RemoteTestRunner.java",
                "line": 683,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                "method": "run",
                "file": "RemoteTestRunner.java",
                "line": 390,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }, {
                "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner",
                "method": "main",
                "file": "RemoteTestRunner.java",
                "line": 197,
                "exact": true,
                "location": ".cp/",
                "version": "?"
              }],
              "commonElementCount": 0
            },
            "loggerFQCN": "f.q.c.n",
            "endOfBatch": false,
            "contextMap": [{
              "key": "MDC.B",
              "value": "B_Value"
            }, {
              "key": "MDC.A",
              "value": "A_Value"
            }],
            "contextStack": ["stack_msg1", "stack_msg2"],
            "source": {
              "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest",
              "method": "testAllFeatures",
              "file": "JSONLayoutTest.java",
              "line": 120
            }
          }
          
          Show
          Gary Gregory added a comment - You should expect the following before 2.0 (ASAP): { "timeMillis": 1, "thread": "MyThreadName", "level": "DEBUG", "loggerName": "a.B", "marker": { "name": "Marker1", "parents": [{ "name": "ParentMarker1" }, { "name": "ParentMarker2" }] }, "message": "Msg", "thrown": { "cause": { "name": "java.lang.NullPointerException", "message": "testNPEx", "localizedMessage": "testNPEx", "extendedStackTrace": [{ "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest", "method": "testAllFeatures", "file": "JSONLayoutTest.java", "line": 122, "exact": false , "location": "test-classes/", "version": "?" }, { "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest", "method": "testLocationOnCompactOnMdcOn", "file": "JSONLayoutTest.java", "line": 298, "exact": false }, { "class": "sun.reflect.NativeMethodAccessorImpl", "method": "invoke0", "line": -2, "exact": false }, { "class": "sun.reflect.NativeMethodAccessorImpl", "method": "invoke", "line": -1, "exact": false }, { "class": "sun.reflect.DelegatingMethodAccessorImpl", "method": "invoke", "line": -1, "exact": false }, { "class": "java.lang.reflect.Method", "method": "invoke", "line": -1, "exact": false }, { "class": "org.junit.runners.model.FrameworkMethod$1", "method": "runReflectiveCall", "file": "FrameworkMethod.java", "line": 47, "exact": false }, { "class": "org.junit.internal.runners.model.ReflectiveCallable", "method": "run", "file": "ReflectiveCallable.java", "line": 12, "exact": false }, { "class": "org.junit.runners.model.FrameworkMethod", "method": "invokeExplosively", "file": "FrameworkMethod.java", "line": 44, "exact": false }, { "class": "org.junit.internal.runners.statements.InvokeMethod", "method": "evaluate", "file": "InvokeMethod.java", "line": 17, "exact": false }, { "class": "org.junit.runners.ParentRunner", "method": "runLeaf", "file": "ParentRunner.java", "line": 271, "exact": false }, { "class": "org.junit.runners.BlockJUnit4ClassRunner", "method": "runChild", "file": "BlockJUnit4ClassRunner.java", "line": 70, "exact": false }, { "class": "org.junit.runners.BlockJUnit4ClassRunner", "method": "runChild", "file": "BlockJUnit4ClassRunner.java", "line": 50, "exact": false }, { "class": "org.junit.runners.ParentRunner$3", "method": "run", "file": "ParentRunner.java", "line": 238, "exact": false }, { "class": "org.junit.runners.ParentRunner$1", "method": "schedule", "file": "ParentRunner.java", "line": 63, "exact": false }, { "class": "org.junit.runners.ParentRunner", "method": "runChildren", "file": "ParentRunner.java", "line": 236, "exact": false }, { "class": "org.junit.runners.ParentRunner", "method": "access$000", "file": "ParentRunner.java", "line": 53, "exact": false }, { "class": "org.junit.runners.ParentRunner$2", "method": "evaluate", "file": "ParentRunner.java", "line": 229, "exact": false }, { "class": "org.junit.internal.runners.statements.RunBefores", "method": "evaluate", "file": "RunBefores.java", "line": 26, "exact": false }, { "class": "org.junit.internal.runners.statements.RunAfters", "method": "evaluate", "file": "RunAfters.java", "line": 27, "exact": false }, { "class": "org.junit.runners.ParentRunner", "method": "run", "file": "ParentRunner.java", "line": 309, "exact": false }, { "class": "org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference", "method": "run", "file": "JUnit4TestReference.java", "line": 50, "exact": false }, { "class": "org.eclipse.jdt.internal.junit.runner.TestExecution", "method": "run", "file": "TestExecution.java", "line": 38, "exact": false }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "runTests", "file": "RemoteTestRunner.java", "line": 467, "exact": false }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "runTests", "file": "RemoteTestRunner.java", "line": 683, "exact": false }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "run", "file": "RemoteTestRunner.java", "line": 390, "exact": false }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "main", "file": "RemoteTestRunner.java", "line": 197, "exact": false }], "commonElementCount": 26 }, "name": "java.io.IOException", "message": "testIOEx", "localizedMessage": "testIOEx", "extendedStackTrace": [{ "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest", "method": "testAllFeatures", "file": "JSONLayoutTest.java", "line": 122, "exact": true , "location": "test-classes/", "version": "?" }, { "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest", "method": "testLocationOnCompactOnMdcOn", "file": "JSONLayoutTest.java", "line": 298, "exact": true , "location": "test-classes/", "version": "?" }, { "class": "sun.reflect.NativeMethodAccessorImpl", "method": "invoke0", "line": -2, "exact": false , "location": "?", "version": "1.7.0_55" }, { "class": "sun.reflect.NativeMethodAccessorImpl", "method": "invoke", "line": -1, "exact": false , "location": "?", "version": "1.7.0_55" }, { "class": "sun.reflect.DelegatingMethodAccessorImpl", "method": "invoke", "line": -1, "exact": false , "location": "?", "version": "1.7.0_55" }, { "class": "java.lang.reflect.Method", "method": "invoke", "line": -1, "exact": false , "location": "?", "version": "1.7.0_55" }, { "class": "org.junit.runners.model.FrameworkMethod$1", "method": "runReflectiveCall", "file": "FrameworkMethod.java", "line": 47, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.internal.runners.model.ReflectiveCallable", "method": "run", "file": "ReflectiveCallable.java", "line": 12, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.model.FrameworkMethod", "method": "invokeExplosively", "file": "FrameworkMethod.java", "line": 44, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.internal.runners.statements.InvokeMethod", "method": "evaluate", "file": "InvokeMethod.java", "line": 17, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner", "method": "runLeaf", "file": "ParentRunner.java", "line": 271, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.BlockJUnit4ClassRunner", "method": "runChild", "file": "BlockJUnit4ClassRunner.java", "line": 70, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.BlockJUnit4ClassRunner", "method": "runChild", "file": "BlockJUnit4ClassRunner.java", "line": 50, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner$3", "method": "run", "file": "ParentRunner.java", "line": 238, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner$1", "method": "schedule", "file": "ParentRunner.java", "line": 63, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner", "method": "runChildren", "file": "ParentRunner.java", "line": 236, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner", "method": "access$000", "file": "ParentRunner.java", "line": 53, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner$2", "method": "evaluate", "file": "ParentRunner.java", "line": 229, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.internal.runners.statements.RunBefores", "method": "evaluate", "file": "RunBefores.java", "line": 26, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.internal.runners.statements.RunAfters", "method": "evaluate", "file": "RunAfters.java", "line": 27, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.junit.runners.ParentRunner", "method": "run", "file": "ParentRunner.java", "line": 309, "exact": true , "location": "junit-4.11.jar", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference", "method": "run", "file": "JUnit4TestReference.java", "line": 50, "exact": true , "location": ".cp/", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit.runner.TestExecution", "method": "run", "file": "TestExecution.java", "line": 38, "exact": true , "location": ".cp/", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "runTests", "file": "RemoteTestRunner.java", "line": 467, "exact": true , "location": ".cp/", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "runTests", "file": "RemoteTestRunner.java", "line": 683, "exact": true , "location": ".cp/", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "run", "file": "RemoteTestRunner.java", "line": 390, "exact": true , "location": ".cp/", "version": "?" }, { "class": "org.eclipse.jdt.internal.junit.runner.RemoteTestRunner", "method": "main", "file": "RemoteTestRunner.java", "line": 197, "exact": true , "location": ".cp/", "version": "?" }], "commonElementCount": 0 }, "loggerFQCN": "f.q.c.n", "endOfBatch": false , "contextMap": [{ "key": "MDC.B", "value": "B_Value" }, { "key": "MDC.A", "value": "A_Value" }], "contextStack": ["stack_msg1", "stack_msg2"], "source": { "class": "org.apache.logging.log4j.core.layout.JSONLayoutTest", "method": "testAllFeatures", "file": "JSONLayoutTest.java", "line": 120 } }
          Hide
          Ralph Goers added a comment -

          Unless you can provide a reason that the contextMap HAS to be like that you can expect my objection.

          Show
          Ralph Goers added a comment - Unless you can provide a reason that the contextMap HAS to be like that you can expect my objection.
          Hide
          Gary Gregory added a comment -

          Simple: If it is not this way, it is not possible to define and use an XML Schema (or JSON Schema) to validate XML (or JSON) events.

          If you use the key values as the tag names, then each different kind of context map will requires a special schema for those kinds of event. I am not considering defining "anything can go in a context map" a useful schema fragment definition.

          Show
          Gary Gregory added a comment - Simple: If it is not this way, it is not possible to define and use an XML Schema (or JSON Schema) to validate XML (or JSON) events. If you use the key values as the tag names, then each different kind of context map will requires a special schema for those kinds of event. I am not considering defining "anything can go in a context map" a useful schema fragment definition.
          Hide
          Ralph Goers added a comment -

          That is hardly surprising since the JSON spec doesn't define a "Map". However, you can find plenty of examples where people are validating Maps using the proper syntax. Remember, in JSON a "Map" is nothing more than an array with a String key and a value (where the value can be one or more objects). I am not in favor of forcing a non-standard JSON syntax for this.

          This has nothing to do with XML. The "standard" way to define a Map in XML is as you have specified it.

          It sounds to me like you are trying to force a square peg in a round hole.

          Show
          Ralph Goers added a comment - That is hardly surprising since the JSON spec doesn't define a "Map". However, you can find plenty of examples where people are validating Maps using the proper syntax. Remember, in JSON a "Map" is nothing more than an array with a String key and a value (where the value can be one or more objects). I am not in favor of forcing a non-standard JSON syntax for this. This has nothing to do with XML. The "standard" way to define a Map in XML is as you have specified it. It sounds to me like you are trying to force a square peg in a round hole.
          Hide
          Gary Gregory added a comment -

          As a reminder, the goal is to be able to ship a single XML Schema and a single JSON Schema to describe an Event.

          Here we are only talking about the context map portion of the schema.

          For XML, the representation of a context map should be along the line of:

          Example 1:

          <ContextMap>
          	<item key="MDC.B" value="B_Value"/>
          	<item key="MDC.A" value="A_Value"/>
          </ContextMap>
          

          Example 2:

          <ContextMap>
          	<item key="MDC.C" value="C_Value"/>
          	<item key="MDC.D" value="D_Value"/>
          </ContextMap>
          

          Both of the above XML documents and all other documents for any key and value can be validated with the same XML Schema, for example:

          <?xml version="1.0" encoding="UTF-8"?>
          <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
          	<xs:element name="item">
          		<xs:complexType>
          			<xs:attribute name="value" use="required"/>
          			<xs:attribute name="key" use="required"/>
          		</xs:complexType>
          	</xs:element>
          	<xs:element name="ContextMap">
          		<xs:complexType>
          			<xs:sequence>
          				<xs:element ref="item" maxOccurs="unbounded"/>
          			</xs:sequence>
          		</xs:complexType>
          	</xs:element>
          </xs:schema>
          

          While the following two documents cannot be validate with the same XML schema. Using XML Schema's "any" is not a solution because that would allow anything at all.

          <ContextMap>
          	<MDC.B>B_Value</MDC.B>
          	<MDC.A>A_Value</MDC.A>
          </ContextMap>
          

          and:

          <ContextMap>
          	<MDC.C>C_Value</MDC.C>
          	<MDC.D>D_Value</MDC.D>
          </ContextMap>
          

          For JSON, there is no W3C standard for JSON itself or any JSON Schema, but I do see references to http://json-schema.org/

          Are you saying that you can write a JSON Schema that can validate both of the following documents while allowing for any key and value?

          "mapName: {
           "MDC.B": "B.Value",
           "MDC.A": "A.Value"
          }
          

          and

          "mapName: {
           "MDC.C": "C.Value",
           "MDC.D": "D.Value"
          }
          

          I'd like to see if so we can ship it!

          Show
          Gary Gregory added a comment - As a reminder, the goal is to be able to ship a single XML Schema and a single JSON Schema to describe an Event. Here we are only talking about the context map portion of the schema. For XML, the representation of a context map should be along the line of: Example 1: <ContextMap> <item key= "MDC.B" value= "B_Value" /> <item key= "MDC.A" value= "A_Value" /> </ContextMap> Example 2: <ContextMap> <item key= "MDC.C" value= "C_Value" /> <item key= "MDC.D" value= "D_Value" /> </ContextMap> Both of the above XML documents and all other documents for any key and value can be validated with the same XML Schema, for example: <?xml version= "1.0" encoding= "UTF-8" ?> <xs:schema xmlns:xs = "http://www.w3.org/2001/XMLSchema" > <xs:element name= "item" > <xs:complexType> <xs:attribute name= "value" use= "required" /> <xs:attribute name= "key" use= "required" /> </xs:complexType> </xs:element> <xs:element name= "ContextMap" > <xs:complexType> <xs:sequence> <xs:element ref= "item" maxOccurs= "unbounded" /> </xs:sequence> </xs:complexType> </xs:element> </xs:schema> While the following two documents cannot be validate with the same XML schema. Using XML Schema's "any" is not a solution because that would allow anything at all. <ContextMap> <MDC.B> B_Value </MDC.B> <MDC.A> A_Value </MDC.A> </ContextMap> and: <ContextMap> <MDC.C> C_Value </MDC.C> <MDC.D> D_Value </MDC.D> </ContextMap> For JSON, there is no W3C standard for JSON itself or any JSON Schema, but I do see references to http://json-schema.org/ Are you saying that you can write a JSON Schema that can validate both of the following documents while allowing for any key and value? "mapName: { "MDC.B": "B.Value", "MDC.A": "A.Value" } and "mapName: { "MDC.C": "C.Value", "MDC.D": "D.Value" } I'd like to see if so we can ship it!
          Hide
          Joern Huxhorn added a comment -

          I'd suggest to use something like

          <ContextMap>
          	<item key="MDC.A">A_Value</item>
          	<item key="MDC.B">B_Value</item>
          </ContextMap>
          

          for XML.

          The difference is subtle. While line-breaks are allowed in XML, an XML parser is normalizing attribute values, changing line-breaks to space. So writing and reading the Thread Context wouldn't be a lossless process currently.

          Regarding JSON, it should definitely look like this:

          "threadContext": {
           "MDC.A": "A.Value",
           "MDC.B": "B.Value"
          }
          

          JSON isn't XML.

          Show
          Joern Huxhorn added a comment - I'd suggest to use something like <ContextMap> <item key= "MDC.A" > A_Value </item> <item key= "MDC.B" > B_Value </item> </ContextMap> for XML. The difference is subtle. While line-breaks are allowed in XML , an XML parser is normalizing attribute values, changing line-breaks to space. So writing and reading the Thread Context wouldn't be a lossless process currently. Regarding JSON, it should definitely look like this: "threadContext": { "MDC.A": "A.Value", "MDC.B": "B.Value" } JSON isn't XML.
          Hide
          Mikael Ståldal added a comment -

          Why are you going to undo the improvement to the context map?

          (I don't care if the context map is named "contextMap", "mdc" or "Properties".)

          Show
          Mikael Ståldal added a comment - Why are you going to undo the improvement to the context map? (I don't care if the context map is named "contextMap", "mdc" or "Properties".)
          Hide
          Mikael Ståldal added a comment -

          marker now looks like this:

           "marker": {
              "name": "Marker1",
              "parents": [{
                "name": "ParentMarker1"
              }, {
                "name": "ParentMarker2"
              }]
            },
          

          perhaps it can be simplified to this:

           "marker": ["Marker1", "ParentMarker1", "ParentMarker2"],
          

          It will then be similar in structure to contextStack.

          Show
          Mikael Ståldal added a comment - marker now looks like this: "marker" : { "name" : "Marker1" , "parents" : [{ "name" : "ParentMarker1" }, { "name" : "ParentMarker2" }] }, perhaps it can be simplified to this: "marker" : [ "Marker1" , "ParentMarker1" , "ParentMarker2" ], It will then be similar in structure to contextStack.
          Hide
          Mikael Ståldal added a comment -

          About JSON schema for the contextMap, can't you just declare it as "type": "object"?

          Show
          Mikael Ståldal added a comment - About JSON schema for the contextMap, can't you just declare it as "type": "object"?
          Hide
          Joern Huxhorn added a comment -

          If you are interested: this is an example of the Lilith JSON.

          Take a look at the encoder/decoder over at GitHub. It's using Jackson (which is available under Apache license).

          I have yet to see any use of JSON Schema in the wild. The usual parsers (Jackson, Boon, GSON) certainly don't support it and if it doesn't support key-value maps like the above then it's useless anyway.

          Show
          Joern Huxhorn added a comment - If you are interested: this is an example of the Lilith JSON. Take a look at the encoder/decoder over at GitHub . It's using Jackson (which is available under Apache license). I have yet to see any use of JSON Schema in the wild. The usual parsers (Jackson, Boon, GSON) certainly don't support it and if it doesn't support key-value maps like the above then it's useless anyway.
          Hide
          Gary Gregory added a comment -

          @Jeorn: The above XML and JSON was produced using (almost) the same Jackson code. This is not committed yet.

          Show
          Gary Gregory added a comment - @Jeorn: The above XML and JSON was produced using (almost) the same Jackson code. This is not committed yet.
          Hide
          Gary Gregory added a comment -

          @Mikael Ståldal : Your example does not cut it, you need to be able to represent multiple inheritance, for example:

                  final Marker cMarker = MarkerManager.getMarker("Marker1");
                  final Marker pMarker1 = MarkerManager.getMarker("ParentMarker1");
                  final Marker pMarker2 = MarkerManager.getMarker("ParentMarker2");
                  final Marker gfMarker = MarkerManager.getMarker("GrandFatherMarker");
                  final Marker gmMarker = MarkerManager.getMarker("GrandMotherMarker");
                  cMarker.addParents(pMarker1);
                  cMarker.addParents(gfMarker);
                  pMarker1.addParents(gmMarker);
                  pMarker1.addParents(gfMarker);
          

          Which I represent as:

            "marker": {
              "name": "Marker1",
              "parents": [{
                "name": "ParentMarker1",
                "parents": [{
                  "name": "GrandMotherMarker"
                }, {
                  "name": "GrandFatherMarker"
                }]
              }, {
                "name": "GrandFatherMarker"
              }]
            },
          
          Show
          Gary Gregory added a comment - @Mikael Ståldal : Your example does not cut it, you need to be able to represent multiple inheritance, for example: final Marker cMarker = MarkerManager.getMarker( "Marker1" ); final Marker pMarker1 = MarkerManager.getMarker( "ParentMarker1" ); final Marker pMarker2 = MarkerManager.getMarker( "ParentMarker2" ); final Marker gfMarker = MarkerManager.getMarker( "GrandFatherMarker" ); final Marker gmMarker = MarkerManager.getMarker( "GrandMotherMarker" ); cMarker.addParents(pMarker1); cMarker.addParents(gfMarker); pMarker1.addParents(gmMarker); pMarker1.addParents(gfMarker); Which I represent as: "marker": { "name": "Marker1", "parents": [{ "name": "ParentMarker1", "parents": [{ "name": "GrandMotherMarker" }, { "name": "GrandFatherMarker" }] }, { "name": "GrandFatherMarker" }] },
          Hide
          Ralph Goers added a comment -

          You don't have to represent multiple inheritence. Instead it could just be

          "marker": "marker1"
          

          with no parents and

          "marker": {"marker1" : ["parent1", "parent2"] }
          

          with parents. If you really want parents then you would do:

          "marker": {"marker1" : ["parent1", {"parent2" : ["grandparent1", "grandparent2"]}, {"parent3" : "grandparent3"}}
          
          Show
          Ralph Goers added a comment - You don't have to represent multiple inheritence. Instead it could just be "marker" : "marker1" with no parents and "marker" : { "marker1" : [ "parent1" , "parent2" ] } with parents. If you really want parents then you would do: "marker" : { "marker1" : [ "parent1" , { "parent2" : [ "grandparent1" , "grandparent2" ]}, { "parent3" : "grandparent3" }}
          Hide
          Ralph Goers added a comment -

          The above should say "if you really want grandparents".

          Show
          Ralph Goers added a comment - The above should say "if you really want grandparents".
          Hide
          Ralph Goers added a comment -

          Gary,

          You seem to want to name all your elements. In the representation I show above the Markers are objects. The Marker can contain no parents, 1 parent or multiple parents. Each parent can in turn have parents. You do this just through a JSON object representation.

          Show
          Ralph Goers added a comment - Gary, You seem to want to name all your elements. In the representation I show above the Markers are objects. The Marker can contain no parents, 1 parent or multiple parents. Each parent can in turn have parents. You do this just through a JSON object representation.
          Hide
          Gary Gregory added a comment -

          I like this latest marker representation. It's easy to read and terse. I'm sure there's a way to do that in Jackson. It is quite powerful now that I've spent some time with it.

          Let do this: I'll commit what I have now which works with all tests passing. This gives us Jackson based XML and JSON listeners and layouts.

          Then we can fiddle with each little element and attribute until we settle upon a final representation.

          My goals here are:

          • Add XML and JSON listeners to Log4j 2.
          • Find the easiest way to make sure that layouts and listeners are in sync. Jackson is great way to do this.
          • Stay away from writing low-level XML/DOM code and JSON parsing code. This would be a lot of code to maintain and forget keeping the layouts and listeners easily in sync.
          Show
          Gary Gregory added a comment - I like this latest marker representation. It's easy to read and terse. I'm sure there's a way to do that in Jackson. It is quite powerful now that I've spent some time with it. Let do this: I'll commit what I have now which works with all tests passing. This gives us Jackson based XML and JSON listeners and layouts . Then we can fiddle with each little element and attribute until we settle upon a final representation. My goals here are: Add XML and JSON listeners to Log4j 2. Find the easiest way to make sure that layouts and listeners are in sync. Jackson is great way to do this. Stay away from writing low-level XML/DOM code and JSON parsing code. This would be a lot of code to maintain and forget keeping the layouts and listeners easily in sync.
          Hide
          Joern Huxhorn added a comment -

          @Gary Ah, ok. The earlier comment from Mikael with the malformed JSON made me a bit nervous. I thought you'd just build that JSON manually which is usually a bad idea. I love Jackson.

          Show
          Joern Huxhorn added a comment - @Gary Ah, ok. The earlier comment from Mikael with the malformed JSON made me a bit nervous. I thought you'd just build that JSON manually which is usually a bad idea. I love Jackson.
          Hide
          Gary Gregory added a comment -

          @Joern Huxhorn : Yeah, previous Log4j 1 and 2 implementation of the JSON (v 2 only) and XML layouts were basically giant toStrings, nasty At least we agree on that!

          Show
          Gary Gregory added a comment - @Joern Huxhorn : Yeah, previous Log4j 1 and 2 implementation of the JSON (v 2 only) and XML layouts were basically giant toStrings, nasty At least we agree on that!
          Hide
          Mikael Ståldal added a comment -

          I did not know that markers support multiple inheritance, since the current documentation says that they don't:

          http://logging.apache.org/log4j/2.x/manual/markers.html
          http://logging.apache.org/log4j/2.x/log4j-api/apidocs/index.html

          But I see now that this has changed in the latest code in SVN. So forget about my comment about markers.

          Show
          Mikael Ståldal added a comment - I did not know that markers support multiple inheritance, since the current documentation says that they don't: http://logging.apache.org/log4j/2.x/manual/markers.html http://logging.apache.org/log4j/2.x/log4j-api/apidocs/index.html But I see now that this has changed in the latest code in SVN. So forget about my comment about markers.
          Hide
          Mikael Ståldal added a comment -

          The latest code from SVN does not seem to work, I get this error when trying to use JSONLayout:

          java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonProcessingException
          at java.lang.Class.getDeclaredMethods0(Native Method)
          at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
          at java.lang.Class.privateGetPublicMethods(Class.java:2651)
          at java.lang.Class.getMethods(Class.java:1467)
          at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.withFactoryMethodAnnotatedBy(PluginBuilder.java:87)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:710)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:665)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:657)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:657)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:334)
          at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:156)
          at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:380)
          at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:427)
          at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:164)
          at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:73)
          at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:35)
          at org.apache.logging.log4j.LogManager.getContext(LogManager.java:268)
          at org.apache.log4j.Logger$PrivateManager.getContext(Logger.java:59)
          at org.apache.log4j.Logger.getLogger(Logger.java:41)

          Show
          Mikael Ståldal added a comment - The latest code from SVN does not seem to work, I get this error when trying to use JSONLayout: java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonProcessingException at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2531) at java.lang.Class.privateGetPublicMethods(Class.java:2651) at java.lang.Class.getMethods(Class.java:1467) at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.withFactoryMethodAnnotatedBy(PluginBuilder.java:87) at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:710) at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:665) at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:657) at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:657) at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:334) at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:156) at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:380) at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:427) at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:164) at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:73) at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:35) at org.apache.logging.log4j.LogManager.getContext(LogManager.java:268) at org.apache.log4j.Logger$PrivateManager.getContext(Logger.java:59) at org.apache.log4j.Logger.getLogger(Logger.java:41)
          Hide
          Gary Gregory added a comment -

          Hm, let's see JsonProcessingException is in jackson-core-2.3.3.jar and I see the following in log4j-core's pom.xml:

              <dependency>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-core</artifactId>
                <optional>true</optional>
              </dependency>
          

          So you need to make jackson-core required in your app's POM.

          Now we need to check if our use of jackson-core is really optional...

          BTW, do you use an XML or JSON configuration file?

          Thank you,
          Gary

          Show
          Gary Gregory added a comment - Hm, let's see JsonProcessingException is in jackson-core-2.3.3.jar and I see the following in log4j-core 's pom.xml : <dependency> <groupId> com.fasterxml.jackson.core </groupId> <artifactId> jackson-core </artifactId> <optional> true </optional> </dependency> So you need to make jackson-core required in your app's POM. Now we need to check if our use of jackson-core is really optional... BTW, do you use an XML or JSON configuration file? Thank you, Gary
          Hide
          Mikael Ståldal added a comment -

          I use an XML configuration file, and JSONLayout.

          I don't use Jackson in my app, so my app's POM doesn't say anything about it. And why should it? The use of Jackson is an implementation detail of Log4j that the user shouldn't need to care about, isn't it?

          (BTW, I use Gradle and not Maven for building my app.)

          Show
          Mikael Ståldal added a comment - I use an XML configuration file, and JSONLayout. I don't use Jackson in my app, so my app's POM doesn't say anything about it. And why should it? The use of Jackson is an implementation detail of Log4j that the user shouldn't need to care about, isn't it? (BTW, I use Gradle and not Maven for building my app.)
          Hide
          Mikael Ståldal added a comment -

          I tried to remove the <optional>true</optional> from jackson-core in log4j-core, just to find out the the same issue applies to jackson-databind.

          After removing <optional>true</optional> from jackson-databind as well, JSONLayout works.

          Show
          Mikael Ståldal added a comment - I tried to remove the <optional>true</optional> from jackson-core in log4j-core, just to find out the the same issue applies to jackson-databind. After removing <optional>true</optional> from jackson-databind as well, JSONLayout works.
          Hide
          Ralph Goers added a comment -

          You don't want to be modifying Log4j's pom.xml. These jars are optional because they are only required if you want support for JSON, and I suppose now also for XML serialization/deserialization. Instead, add the jars to your project's pom.xml as dependencies.

          Show
          Ralph Goers added a comment - You don't want to be modifying Log4j's pom.xml. These jars are optional because they are only required if you want support for JSON, and I suppose now also for XML serialization/deserialization. Instead, add the jars to your project's pom.xml as dependencies.
          Hide
          Mikael Ståldal added a comment -

          Now I get this problem instead, probably not related to JSONLayout:

          java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.web.Log4jWebInitializerImpl
          at java.lang.Class.forName0(Native Method) ~[?:1.7.0_51]
          at java.lang.Class.forName(Class.java:270) ~[?:1.7.0_51]
          at org.apache.logging.log4j.core.util.Loader.initializeClass(Loader.java:272) ~[log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:483) ~[log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:595) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:145) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.Log4jLogEvent.<init>(Log4jLogEvent.java:121) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.Log4jLogEvent.<init>(Log4jLogEvent.java:97) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.impl.DefaultLogEventFactory.createEvent(DefaultLogEventFactory.java:49) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:103) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]
          at org.apache.logging.slf4j.Log4jLogger.log(Log4jLogger.java:374) [log4j-slf4j-impl-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT]

          I have declared dependency to log4j-web. I think that the class org.apache.logging.log4j.web.Log4jWebInitializerImplis found, but fails to initialize for some reason.

          Show
          Mikael Ståldal added a comment - Now I get this problem instead, probably not related to JSONLayout: java.lang.NoClassDefFoundError: Could not initialize class org.apache.logging.log4j.web.Log4jWebInitializerImpl at java.lang.Class.forName0(Native Method) ~ [?:1.7.0_51] at java.lang.Class.forName(Class.java:270) ~ [?:1.7.0_51] at org.apache.logging.log4j.core.util.Loader.initializeClass(Loader.java:272) ~ [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:483) ~ [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:595) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:145) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.Log4jLogEvent.<init>(Log4jLogEvent.java:121) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.Log4jLogEvent.<init>(Log4jLogEvent.java:97) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.impl.DefaultLogEventFactory.createEvent(DefaultLogEventFactory.java:49) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:103) [log4j-core-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] at org.apache.logging.slf4j.Log4jLogger.log(Log4jLogger.java:374) [log4j-slf4j-impl-2.0-rc2-SNAPSHOT.jar:2.0-rc2-SNAPSHOT] I have declared dependency to log4j-web. I think that the class org.apache.logging.log4j.web.Log4jWebInitializerImplis found, but fails to initialize for some reason.
          Hide
          Mikael Ståldal added a comment -

          Then you need to clearly document when those Jackson dependencies are needed, since you cannot expect a regular Log4j user to know that.

          Show
          Mikael Ståldal added a comment - Then you need to clearly document when those Jackson dependencies are needed, since you cannot expect a regular Log4j user to know that.
          Hide
          Joern Huxhorn added a comment -

          I suppose moving the XML- and JSON-layouts into respective modules, each with proper non-optional dependencies, isn't an option? Not sure if they should really be considered core functionality of log4j.

          <optional> is often a "dependency smell" indicating insufficient modularization. A counter-example would be an optional woodstox dependency that would replace the Java StAX implementation with the (faster) woodstox one if present. But the code would still work even without that woodstox dependency, making the dependency truly optional. The code would simply have worse performance.

          Explaining that an "log4j2-json-layout" dependency needs to be added if the JSON layout is desired is easier to grasp than "Oh, that error? That means you need to add these two jackson dependencies.", probably on StackOverflow. People (at least some) tend to not read documentation. And those are enough to create unnecessary noise/support overhead on the user mailinglist.

          Show
          Joern Huxhorn added a comment - I suppose moving the XML- and JSON-layouts into respective modules, each with proper non-optional dependencies, isn't an option? Not sure if they should really be considered core functionality of log4j. <optional> is often a "dependency smell" indicating insufficient modularization. A counter-example would be an optional woodstox dependency that would replace the Java StAX implementation with the (faster) woodstox one if present. But the code would still work even without that woodstox dependency, making the dependency truly optional. The code would simply have worse performance. Explaining that an "log4j2-json-layout" dependency needs to be added if the JSON layout is desired is easier to grasp than "Oh, that error? That means you need to add these two jackson dependencies.", probably on StackOverflow. People (at least some) tend to not read documentation. And those are enough to create unnecessary noise/support overhead on the user mailinglist.
          Hide
          Ralph Goers added a comment -

          While I agree in principal, what is awkward about that is that while the JSONConfiguration and JSONLayout both require Jackson, the XMLConfiguration does not while XMLLayout now also requires Jackson, so the separate module would contain JSONConfiguration, JSONLayout and XMLLayout, which seems a bit odd to include in a log4j-json jar.

          Show
          Ralph Goers added a comment - While I agree in principal, what is awkward about that is that while the JSONConfiguration and JSONLayout both require Jackson, the XMLConfiguration does not while XMLLayout now also requires Jackson, so the separate module would contain JSONConfiguration, JSONLayout and XMLLayout, which seems a bit odd to include in a log4j-json jar.
          Hide
          Matt Sicker added a comment -

          You're just supposed to add jackson-databind to your project's dependencies to get support for JSON in log4j. No need to edit the log4j pom.xml files.

          Show
          Matt Sicker added a comment - You're just supposed to add jackson-databind to your project's dependencies to get support for JSON in log4j. No need to edit the log4j pom.xml files.
          Hide
          Mikael Ståldal added a comment -

          But how is the poor Log4j user supposed to know that he/she needs to include Jackson dependencies in his/her project in order to get JSONLayout to work? That user may not even know what Jackson is.

          Show
          Mikael Ståldal added a comment - But how is the poor Log4j user supposed to know that he/she needs to include Jackson dependencies in his/her project in order to get JSONLayout to work? That user may not even know what Jackson is.
          Hide
          Gary Gregory added a comment -

          Another path would be to redo XML and JSON configuration to be based on the same common Jackson code. All of the XML/JSON layout/listeners are all based on the same Jackson code.

          The side effect is that Jackson would be required, unless you'd programmatically configure Log4j, which I am not sure is possible.

          Show
          Gary Gregory added a comment - Another path would be to redo XML and JSON configuration to be based on the same common Jackson code. All of the XML/JSON layout/listeners are all based on the same Jackson code. The side effect is that Jackson would be required, unless you'd programmatically configure Log4j, which I am not sure is possible.
          Hide
          Joern Huxhorn added a comment -

          @Ralph Ah, I see. I wasn't aware that JSON configuration exists.

          Show
          Joern Huxhorn added a comment - @Ralph Ah, I see. I wasn't aware that JSON configuration exists.
          Hide
          Ralph Goers added a comment -

          Gary, I would not be in favor of making Jackson a required dependency. IMO, users need to be able to have a functional Log4j that is able to run with only the api and core jars.

          Show
          Ralph Goers added a comment - Gary, I would not be in favor of making Jackson a required dependency. IMO, users need to be able to have a functional Log4j that is able to run with only the api and core jars.
          Hide
          Ralph Goers added a comment -

          Mikael, I would say that the Javadoc for the component needs to call it out plus we should have a page that lists the dependencies and when they are required.

          Show
          Ralph Goers added a comment - Mikael, I would say that the Javadoc for the component needs to call it out plus we should have a page that lists the dependencies and when they are required.
          Hide
          Mikael Ståldal added a comment -

          This is not properly fixed, in 2.0 it looks like this:

            "contextMap" : [ {
              "key" : "foo",
              "value" : "FOO"
            }, {
              "key" : "bar",
              "value" : "BAR"
            } ]
          
          Show
          Mikael Ståldal added a comment - This is not properly fixed, in 2.0 it looks like this: "contextMap" : [ { "key" : "foo" , "value" : "FOO" }, { "key" : "bar" , "value" : "BAR" } ]
          Hide
          Mikael Ståldal added a comment -

          Still not fixed in 2.2.

          Show
          Mikael Ståldal added a comment - Still not fixed in 2.2.
          Hide
          Mikael Ståldal added a comment -

          Should we close this as Won't Fix since changing the format now will break backwards compatibility?

          Show
          Mikael Ståldal added a comment - Should we close this as Won't Fix since changing the format now will break backwards compatibility?
          Hide
          Mikael Ståldal added a comment -

          Changing this now would break backwards compatibility.

          Show
          Mikael Ståldal added a comment - Changing this now would break backwards compatibility.
          Hide
          Ralph Goers added a comment -

          Somehow I missed that JSONLayout is generating invalid ContextMap data, despite my saying that I objected to this format. This needs to be corrected. I don't really care if it breaks compatibility. What it is doing now is, and always has been, wrong.

          Show
          Ralph Goers added a comment - Somehow I missed that JSONLayout is generating invalid ContextMap data, despite my saying that I objected to this format. This needs to be corrected. I don't really care if it breaks compatibility. What it is doing now is, and always has been, wrong.
          Hide
          Mikael Ståldal added a comment -

          Nice if we can finally fix this.

          Show
          Mikael Ståldal added a comment - Nice if we can finally fix this.
          Hide
          Matt Sicker added a comment -

          Agreed on fixing.

          Show
          Matt Sicker added a comment - Agreed on fixing.
          Hide
          Mikael Ståldal added a comment -

          This is still not fixed.

          Show
          Mikael Ståldal added a comment - This is still not fixed.
          Hide
          Remko Popma added a comment -

          Since the same code is used for XML and JSON, whoever fixed this should take care that the XML format for the context map stays as is, with key="key" and value="value", while JSON and YAML have "key": "value". (Don't break the XML layout when fixing the JSON layout.)

          Show
          Remko Popma added a comment - Since the same code is used for XML and JSON, whoever fixed this should take care that the XML format for the context map stays as is, with key="key" and value="value", while JSON and YAML have "key": "value". (Don't break the XML layout when fixing the JSON layout.)
          Hide
          Gary Gregory added a comment -

          Don't forget that there is YAML as well.

          Show
          Gary Gregory added a comment - Don't forget that there is YAML as well.
          Hide
          Remko Popma added a comment -

          Just to double-check: this is not a showstopper for the 2.6 release, is it?

          Show
          Remko Popma added a comment - Just to double-check: this is not a showstopper for the 2.6 release, is it?
          Hide
          Mikael Ståldal added a comment -

          I think that we should consider at least the YAML layout as a showstopper since it is new. It would be quite silly to release a new layout with incorrect behaviour and then change it in a later release.

          Show
          Mikael Ståldal added a comment - I think that we should consider at least the YAML layout as a showstopper since it is new. It would be quite silly to release a new layout with incorrect behaviour and then change it in a later release.
          Hide
          Remko Popma added a comment -

          Quick summary of the problem

          XmlLayout, JsonLayout and YamlLayout have a shared code base but for the thread context we want to output XML in a different format from JSON/YAML.

          Current JSON format:

            "contextMap" : [ {
              "key" : "foo",
              "value" : "FOO"
            }, {
              "key" : "bar",
              "value" : "BAR"
            } ]
          

          This is OK for XML (and has the advantage the document structure can be captures in a schema) but is not desirable for JSON & YAML where the natural mapping is something like:

          "contextMap": {
           "foo": "FOO",
           "bar": "BAR"
          }
          

          Quick analysis of how this could be implemented
          (I'm new to Jackson, so below is just guess work)

          • context map output is created by ListOfMapEntrySerializer. This class is currently used for all Jackson-generated layouts (XML, JSON, YAML). To keep XML the same but change JSON and YAML we need to create a different serializer class for JSON, YAML.
          • ListOfMapEntrySerializer creates an array of MapEntry objects to output [{"key" : "xxxxx", "value" : "yyyyy"}]. Instead, the new serializer should probably simply call something like the below because the key is not fixed:
          JsonGenerator jgen;
          jgen.writeStartObject();
          for (Entry<String, String> entry : map.entrySet()) {
              jgen.writeStringField(entry.getKey(), entry.getValue());
          }
          jgen.writeEndObject();
          
          • LogEventMixIn#getContextMap has the annotation that determines which serializer to use. Looks like we need to separate this into XmlLogEventMixIn and JsonYamlLogEventMixIn so we can annotate the getContextMap method with the new serializer.
          • The mix-in is installed by Initializers.SetupContextInitializer, which is invoked by all Log4jXmlModule, Log4jYamlModule and Log4jJsonModule: we may need to split this for the separate layouts
          Show
          Remko Popma added a comment - Quick summary of the problem XmlLayout, JsonLayout and YamlLayout have a shared code base but for the thread context we want to output XML in a different format from JSON/YAML. Current JSON format: "contextMap" : [ { "key" : "foo" , "value" : "FOO" }, { "key" : "bar" , "value" : "BAR" } ] This is OK for XML (and has the advantage the document structure can be captures in a schema) but is not desirable for JSON & YAML where the natural mapping is something like: "contextMap" : { "foo" : "FOO" , "bar" : "BAR" } Quick analysis of how this could be implemented (I'm new to Jackson, so below is just guess work) context map output is created by ListOfMapEntrySerializer . This class is currently used for all Jackson-generated layouts (XML, JSON, YAML). To keep XML the same but change JSON and YAML we need to create a different serializer class for JSON, YAML. ListOfMapEntrySerializer creates an array of MapEntry objects to output [{"key" : "xxxxx", "value" : "yyyyy"}] . Instead, the new serializer should probably simply call something like the below because the key is not fixed: JsonGenerator jgen; jgen.writeStartObject(); for (Entry< String , String > entry : map.entrySet()) { jgen.writeStringField(entry.getKey(), entry.getValue()); } jgen.writeEndObject(); LogEventMixIn#getContextMap has the annotation that determines which serializer to use. Looks like we need to separate this into XmlLogEventMixIn and JsonYamlLogEventMixIn so we can annotate the getContextMap method with the new serializer. The mix-in is installed by Initializers.SetupContextInitializer , which is invoked by all Log4jXmlModule, Log4jYamlModule and Log4jJsonModule: we may need to split this for the separate layouts
          Hide
          Gary Gregory added a comment -

          Also maybe the default Jackson behavior does what you want. I'm not sure that the new JSON format can be represented with JSONSchema.

          Show
          Gary Gregory added a comment - Also maybe the default Jackson behavior does what you want. I'm not sure that the new JSON format can be represented with JSONSchema.
          Hide
          Remko Popma added a comment -

          Can you clarify what you mean by the "default Jackson behavior"? We don't need to create a custom serializer? What should we do instead?

          Show
          Remko Popma added a comment - Can you clarify what you mean by the "default Jackson behavior"? We don't need to create a custom serializer? What should we do instead?
          Hide
          Remko Popma added a comment -

          Did you mean we should use these?

          com.fasterxml.jackson.databind.deser.std.MapDeserializer;
          com.fasterxml.jackson.databind.ser.std.MapSerializer;
          
          Show
          Remko Popma added a comment - Did you mean we should use these? com.fasterxml.jackson.databind.deser.std.MapDeserializer; com.fasterxml.jackson.databind.ser.std.MapSerializer;
          Hide
          Remko Popma added a comment -

          The answer to that was: no, just don't specify any serializer/deserializer.

          Show
          Remko Popma added a comment - The answer to that was: no, just don't specify any serializer/deserializer.
          Hide
          Remko Popma added a comment -

          Fixed in master to

          "contextMap": {
           "foo": "FOO",
           "bar": "BAR"
          }
          
          Show
          Remko Popma added a comment - Fixed in master to "contextMap" : { "foo" : "FOO" , "bar" : "BAR" }
          Hide
          Mikael Ståldal added a comment -

          Thanks!

          Show
          Mikael Ståldal added a comment - Thanks!

            People

            • Assignee:
              Remko Popma
              Reporter:
              Mikael Ståldal
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development