Affects Version/s: None
Fix Version/s: master (9.0)
This task focuses on providing gradle-based build equivalent for Lucene and Solr (on master branch). See notes below on why this respin is needed.
The code lives on gradle-master branch. It is kept with sync with master. Try running the following to see an overview of helper guides concerning typical workflow, testing and ant-migration helpers:
A list of items that needs to be added or requires work. If you'd like to work on any of these, please add your name to the list. Once you have a patch/ pull request let me (dweiss) know - I'll try to coordinate the merges.
- Apply forbiddenAPIs
- Generate hardware-aware gradle defaults for parallelism (count of workers and test JVMs).
- Fail the build if --tests filter is applied and no tests execute during the entire build (this allows for an empty set of filtered tests at single project level).
- Port other settings and randomizations from common-build.xml
- Configure security policy/ sandboxing for tests.
- test's console output on -Ptests.verbose=true
- add a :helpDeps explanation to how the dependency system works (palantir plugin, lockfile) and how to retrieve structured information about current dependencies of a given module (in a tree-like output).
- jar checksums, jar checksum computation and validation. This should be done without intermediate folders (directly on dependency sets).
- verify min. JVM version and exact gradle version on build startup to minimize odd build side-effects
- Repro-line for failed tests/ runs.
- add a top-level README note about building with gradle (and the required JVM).
- add an equivalent of 'validate-source-patterns' (check-source-patterns.groovy) to precommit.
- add an equivalent of 'rat-sources' to precommit.
- add an equivalent of 'check-example-lucene-match-version' (solr only) to precommit.
- javadoc compilation
Hard-to-implement stuff already investigated:
Printing console output of failed tests. There doesn't seem to be any way to do this in a reasonably efficient way. There are onOutput listeners but they're slow to operate and solr tests emit tons of output so it's an overkill.
- (LUCENE-9120) Tests working with security-debug logs or other JVM-early log output. Gradle's test runner works by redirecting Java's stdout/ syserr so this just won't work. Perhaps we can spin the ant-based test runner for such corner-cases.
Of lesser importance:
- Add an equivalent of 'documentation-lint" to precommit.
- Do not require files to be committed before running precommit. (staged files are fine).
- add rendering of javadocs (gradlew javadoc)
- identify and port various "regenerate" tasks from ant builds (javacc, precompiled automata, etc.)
- Add Solr packaging for docs/* (see TODO in packaging/build.gradle; currently XSLT...)
- I didn't bother adding Solr dist/test-framework to packaging (who'd use it from a binary distribution?
- There is some python execution in check-broken-links and check-missing-javadocs, not sure if it's been ported
- Precommit doesn't catch unused imports
- Attach javadocs to maven publications.
- Add test 'beasting' (rerunning the same suite multiple times). I'm afraid it'll be difficult to run it sensibly because gradle doesn't offer cwd separation for the forked test runners.
- if you diff solr packaged distribution against ant-created distribution there are minor differences in library versions and some JARs are excluded/ moved around. I didn't try to force these as everything seems to work (tests, etc.) – perhaps these differences should be fixed in the ant build instead.
- Fill in POM details in gradle/defaults-maven.gradle so that they reflect the previous content better (dependencies aside).
- Add any IDE integration layers that should be added (I use IntelliJ and it imports the project out of the box, without the need for any special tuning).
Note: this builds on the work done by Mark Miller and Cao Mạnh Đạt but also applies lessons learned from those two efforts:
- Do not try to do too many things at once. If we deviate too far from master, the branch will be hard to merge.
- Do everything in baby-steps and add small, independent build fragments replacing the old ant infrastructure.
- Try to engage people to run, test and contribute early. It can't be a one-man effort. The more people understand and can contribute to the build, the more healthy it will be.