[LUCENE-10348] Make stopwords resources from analyzers modules visible to ClasspathResourceLoader and ModuleResourceLoader

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.1, 10.0 (main)
• Component/s: core/other
• Lucene Fields: New
• Assignee: Uwe Schindler
• Reporter: Uwe Schindler
• Created: 03/Jan/22 11:00
• Updated: 28/Aug/22 16:33
• Resolved: 04/Jan/22 14:42

Description

This is a followup after LUCENE-10335:

In LUCENE-10335 we fixed all Analyzers static constructors to not rely on the analysis modules to export stopwords files. Nevertheless, it is currently impossible to use CustomAnalyzer and refer to stopwords files in the analyzers module, because those are hidden, rmuir showed some examples of how this is uesed. This would also make it impossible for Solr to reuse the resources files!

Basically we should make the stopwords files (and other resources) visible. We have to options:

• open all packages with resources unconditionally
• open all packages with resources only to module "org.apache.lucene.core"

I prefer the second one. The problem with Java's module system is the following: If you open a package, it is also open for (deep) reflection, which we don't want. So my proposal is to only open those packages only to lucene core, so the ModuleResourceLoader and ClasspathResourceLoader can see those resources (and Solr would be able to delegate to these resource loaders if needed).

I will adds a test in the distribution.tests package that verifies that resources in all analysis packages are open.

Issue Links

• Discovered while testing: LUCENE-10335 IOUtils.getDecodingReader(Class<?>, String) is broken with modules
• is required by: LUCENE-10352 Convert TestAllAnalyzersHaveFactories and TestRandomChains to a global integration test and discover classes to check from module system
• links to: GitHub Pull Request #581