Details
Major Description
Hi all,
I apologize in advance if I haven't adjusted this issue properly.
Short description:
I have deployed Trino with ranger-trino-plugin and I wanted to use ElasticSearch (7.10.2) as a place where I want to store the audit. When I configured ranger-admin to use elasticsearch (audit_store=elasticsearch and all other parameters audit_elasticsearch_*) I started getting errors in the catalina.out: java.lang.NoSuchFieldError: LUCENE_8_5_1. As I increased the version of Lucena, it was written in the logs that an even higher version was needed. So in the end, I moved it to 8.11.3 and 8.4.0 for lucene-spatial since it is the latest.
After it was fixed, I tried to use https for elasticsearch protocol (audit_elasticsearch_protocol) however, it always showed that ranger-admin use http instead of https. I show in code that audit_elasticsearch_protocol is not configured well.
As soon as it done, ranger admin successfully created ES index. However, I need to move from MiscUtil.toDate to MiscUtil.toLocalDate for evtTime "column" since I was getting error: Error converting value to date. Value = 2024-05-13T13:08:47.905Z
As soon as I fixed it, I found an error in Trino that the plugin couldn't insert data into elasticsearch. After I upgraded httpcomponents bug-fix version, it's started inserting data.
I opened PR with the fix 2.4.0 version, do I need to do the same on the master branch?
PR: https://github.com/apache/ranger/pull/314/files
1. Lucene version - fixed problem with writing data to ElasticSearch
Error: java.lang.NoSuchFieldError: LUCENE_8_5_1
I tried to change minor version one by one, but only latest version fit for me.
Changes:
- agents-audit/pom.xml: 311
- pom.xml: 241
2. Elastic search protocol - fixed problem with changing protocol
Even though I changed ranger.audit.elasticsearch.protocol from http to https, audit plugin still using http protocol.
Changes:
- security-admin/scripts/ranger-admin-site-template.xml: 167-170
- security-admin/scripts/setup.sh: 79, 794-797
- security-admin/scripts/upgrade_admin.py: 116
- security-admin/src/main/resources/conf.dist/ranger-admin-site.xml: 53-57
- security-admin/src/test/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsServiceTest.java: 56
3. Audit plugin - cannot write audit to ES
Error: bootstrap method initialization exception
After changing the version of httpcomponents I started seeing audit
Changes:
- pom.xml: 137, 138, 140
4. Ranger admin console - Audit show 1-1-1970
Erro: Error converting value to date. Value = 2024-05-13T13:08:47.905Z
Even though evtTime was ok in ElasticSearch, ranger couldn't show it on GUI.
Changes:
- security-admin/src/main/java/org/apache/ranger/elasticsearch/ElasticSearchAccessAuditsService.java: 260
- security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java: 239
