Blocker Description
Tracing for v2 apis will eagerly consume the contentstream to find a good name for the span. the assumption here is that the operations are cached so the stream ca be consumed early, but that is not the case for the SecurityConfHandler and possibly other parts of the code.
With tracing enabled you cannot add a user, a role, etc. Admin UI Security is broken too.
The clash is between
- V2HttpCall solrReq.getCommands
- SecurityConfHandler CommandOperation.readCommands(req.getContentStreams()..)
relevant stacktrace
2023-08-29 19:13:57.591 INFO (qtp605101809-21) [t:750564258779a1605867f784ee33f78a] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/security/authorization params={wt=json&_=1693361587976} status=0 QTime=0
2023-08-29 19:13:57.613 ERROR (qtp605101809-27) [t:b3ab14d057a292fb71ac727c5d871156] o.a.s.h.RequestHandlerBase Server exception => java.lang.RuntimeException: The JSON must be an Object of the form {"command": {...},...
at org.apache.solr.common.util.CommandOperation.parse(CommandOperation.java:275)
java.lang.RuntimeException: The JSON must be an Object of the form {"command": {...},...
at org.apache.solr.common.util.CommandOperation.parse(CommandOperation.java:275) ~[?:?]
at org.apache.solr.common.util.CommandOperation.readCommands(CommandOperation.java:354) ~[?:?]
at org.apache.solr.common.util.CommandOperation.readCommands(CommandOperation.java:327) ~[?:?]
at org.apache.solr.handler.admin.SecurityConfHandler.doEdit(SecurityConfHandler.java:116) ~[?:?]
at org.apache.solr.handler.admin.SecurityConfHandler.handleRequestBody(SecurityConfHandler.java:89) ~[?:?]
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224) ~[?:?]
at org.apache.solr.api.ApiBag$ReqHandlerToApi.call(ApiBag.java:362) ~[?:?]
at org.apache.solr.api.V2HttpCall.handleAdmin(V2HttpCall.java:438) ~[?:?]
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:870) ~[?:?]
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:539) ~[?:?]
at org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:248) ~[?:?]
at org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:215) ~[?:?]
at org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:241) ~[?:?]
at org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:211) ~[?:?]
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209) ~[?:?]
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192) ~[?:?]
I ran into this a few times and could not pinpoint it until now. the reason this does not trigger on the alway-on tracer is that this is hidden behind a 'TraceUtils.ifNotNoop' check.
Also, I think this might be a problem on 9.x too, not completely sure yet.