[SOLR-12021] Utils.fromJSONResource(...) and ValidatingJsonMap.parse(...) open (resourse) InputStreams they don't close

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.3, 8.0
• Assignee: Chris M. Hostetter
• Reporter: Chris M. Hostetter
• Created: 22/Feb/18 23:08
• Updated: 08/Jun/19 15:16
• Resolved: 23/Feb/18 18:53

Description

A recurring theme that pops up now and then in non-reproducible test failures is for an error like this to show up in the log files when a core/collection is being created...

   [junit4]   2> 674805 ERROR (qtp1337497394-5597) [n:127.0.0.1:55380_np_zxw c:collection2 s:shard6  x:collection2_shard6_replica_n29] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'collection2_shard6_replica_n29': Unable to create core [collection2_shard6_replica_n29] Caused by: invalid API spec: apispec/core.config.Commands.json
...
   [junit4]   2> Caused by: java.lang.RuntimeException: invalid API spec: apispec/core.config.Commands.json
   [junit4]   2>        at org.apache.solr.common.util.ValidatingJsonMap.parse(ValidatingJsonMap.java:318)
   [junit4]   2>        at org.apache.solr.common.util.Utils.lambda$getSpec$0(Utils.java:427)
   [junit4]   2>        at org.apache.solr.api.Api.getSpec(Api.java:65)
   [junit4]   2>        at org.apache.solr.api.ApiBag.register(ApiBag.java:73)
   [junit4]   2>        at org.apache.solr.core.PluginBag.put(PluginBag.java:217)
   [junit4]   2>        at org.apache.solr.core.PluginBag.init(PluginBag.java:274)
   [junit4]   2>        at org.apache.solr.core.RequestHandlers.initHandlersFromConfig(RequestHandlers.java:130)
   [junit4]   2>        at org.apache.solr.core.SolrCore.<init>(SolrCore.java:966)

I believe that the root cause of these types of failures is that these API spec files are being loaded with ValidatingJsonMap.class.getClassLoader().getResourceAsStream(resourceName) – but nothing is ever closing the stream, so it can eventually (depending on what test classes run in each JVM and how many files they try to open like this) cause the JVM to hit the ulimit for open file handles – but that specific cause of the failure is never reported, because ClassLoader.getResourseAsStream(...) is explicity designed to swallow any IOExceptions encountered and just returns "null" (which triggers the "invalid API spec: " thrown by ValidatingJsonMap)

---

ValidatingJsonMap.parse(...) and the very similarly looking Utils.fromJSONResource(...) should be modified to:

• ensure they always close the streams they open
• use ClassLoader.getResource() + URL.openConnection() so that they can properly log if/when an underlying IOException occurs w/details

Issue Links

• is related to: SOLR-12386 Test fails for "Can't find resource" for files in the _default configset