[LUCENE-10255] Fully embrace the java module system - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Patch Available
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
None

Lucene Fields:

New

Description

I've experimented a bit trying to move the code to the JMS. It is surprisingly difficult... A PoC that almost passes all checks is here:
-https://github.com/dweiss/lucene/tree/jms-
https://github.com/dweiss/lucene/tree/jms2

Here are my conclusions so far:

The JMS and gradle add a lot of complexity (this applies to any higher-level tooling, including IDEs, I think). For starters, modules have to be JARs. The effect of this is that what was previously a set of directories from dependencies now has to be a JAR. What was previously an incremental update of a single .class file now ripples throughout the build recreating module JARs (ZIPs!)... I didn't realize it at first, but it's a costly thing to do. I'm not even sure how IDEs handle this issue. (yes, true because gradle splits module resources and classes into separate folders, rendering them unusable as an expanded module).
A Java module contains metadata (such as the module version or main class) that is completely detached from any source file. These things live in a class bytecode of the compiled module-info; interestingly, there is no source-level way to specify it - these class attributes are injected by the 'jar' tool. Gradle has some fancy on-the-fly asm conversion filter that injects it.
Dependencies between modules will effectively live in two places: in gradle build files and in module-info files. ~~And they can go out of sync, although it's probably easy to catch (since javac would complain about missing classes during compilation, even if they're in module path).~~ (with separate module and classpath configurations there is a possibility to verify the consistency).
Probably the biggest challenge (not covered in the PoC) are with our custom javadoc and ecj linter tasks - they see the module-info.java and can't cope with it. At the same time, there is no easy way to exclude that one particular file: ecj would have to accept a full set of sources (command argument limit will be a problem), javac can accept a full set of java sources (external file) but then it doesn't copy doc-files properly anymore (this is probably easier to fix).
There are differences at runtime that are hard to anticipate - for example resource lookups via class loader no longer work (I fixed this in Luke).
We will have to rethink the long-term strategy of how white-box tests work. There are some guidelines here but all of them have some cons (IDEs being confused). https://docs.gradle.org/current/userguide/java_testing.html#sec:java_testing_modular
it's pretty much impossible to exclude transitive dependencies from modules we depend on - if they're not compile-time only (static) requirements, they will have to be present on module path.
supporting modules may or may not work in your IDE.