[SOLR-12859] DocExpirationUpdateProcessorFactory does not work with BasicAuth (and other auth plugins that use PKI)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.5
• Fix Version/s: 8.5, 9.0
• Assignee: Chris M. Hostetter
• Reporter: Varun Thacker
• Created: 12/Oct/18 22:10
• Updated: 24/Mar/20 12:46
• Resolved: 22/Jan/20 17:58

Description

DocExpirationUpdateProcessorFactory initiates it's periodic delete requests in a way that caused the PKIAuthenticationPlugin to treat them as requests from un-authenticated users, causing them to be rejected when forwarded to other nodes.

Allthough there are some open questions as to wether or not the overall design of how/where/when PKIAuthenticationPlugin recognizes a "server initiated" NODE_IS_USER requests (vs how/where/when it should recognize a server initiated request) this issue took a narrow focus of change ONLY the DocExpirationUpdateProcessorFactory to "fix" the way it generated it's delete request to ensure PKIAuthenticationPlugin would recognize these as NODE_IS_USER.

---

 

I setup a cluster with basic auth and then wanted to use Solr's TTL feature ( DocExpirationUpdateProcessorFactory ) to auto-delete documents.

 

Turns out it doesn't work when Basic Auth is enabled. I get the following stacktrace from the logs

2018-10-12 22:06:38.967 ERROR (autoExpireDocs-42-thread-1) [   ] o.a.s.u.p.DocExpirationUpdateProcessorFactory Runtime error in periodic deletion of expired docs: Async exception during distributed update: Error from server at http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6: require authentication



request: http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F192.168.0.8%3A8983%2Fsolr%2Fgettingstarted_shard1_replica_n2%2F&wt=javabin&version=2
org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException: Async exception during distributed update: Error from server at http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6: require authentication



request: http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F192.168.0.8%3A8983%2Fsolr%2Fgettingstarted_shard1_replica_n2%2F&wt=javabin&version=2
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:964) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1976) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:182) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80) ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at org.apache.solr.update.processor.DocExpirationUpdateProcessorFactory$DeleteExpiredDocsRunnable.run(DocExpirationUpdateProcessorFactory.java:419) [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_112]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_112]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_112]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

 

Issue Links

• is related to: SOLR-12860 MetricsHistoryHandler does not work with BasicAuth
• relates to: SOLR-14204 Rethink PKIs hueristic for dealing with 'isSolrThread()'