[KNOX-1091] Knox Audit Logging - duplicate correlation ids

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 1.0.0
• Fix Version/s: 1.1.0
• Component/s: Server
• Labels: audit
• Assignee: Kevin Risden
• Reporter: Kevin Risden
• Created: 31/Oct/17 18:53
• Updated: 12/Apr/22 06:19
• Resolved: 21/Mar/18 17:13

Description

From the Knox User list thread: "Multiple topology audit logging", it came to my attention that Knox seems to be logging duplicate correlation ids. Separating out the topic specifically here to dig a bit deeper.

While looking at our Knox audit logs (Knox 0.9 on HDP 2.5) the "correlation id" doesn't seem to be unique across requests. Is this to be expected? Here is a snippet (anonymized):

grep 7557c91b-2a48-4e09-aefc-44e9892372da /var/knox/gateway-audit.log

17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE||||access|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE1/ID1//|unavailable|Request method: GET
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||authentication|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Groups: []
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|unavailable|Request method: GET
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||dispatch|uri|http://WEBHBASE.example.com:8084/NAMESPACE2:TABLE2/multiget?doAs=USER1&row=ID2%2Fd%3Araw|success|Response status: 200
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE|USER1|||access|uri|/gateway/HADOOPPROD/hbase/NAMESPACE2:TABLE2/multiget?row=ID2%2fd%3araw&|success|Response status: 200
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE||||authentication|principal|USER2|failure|LDAP authentication failed.
17/10/10 12:50:09 ||7557c91b-2a48-4e09-aefc-44e9892372da|audit|WEBHBASE||||access|uri|/gateway/HADOOPTEST/hbase/hbase/NAMESPACE1:TABLE2/ID1//|success|Response status: 401

The things to highlight here for the same correlation id:

• different topologies are being used
• different uris are being used
• different users are being used

Some of the things that we have configured that could impact results:

• authentication caching
• multiple Knox servers
• load balancer in front of Knox

Issue Links

• causes: KNOX-2730 Fix missing Correlation id in audit logs