Lucene came with several tests to ensure backward compatibility with indexes, and included many zipped index archives which are used to ensure Lucene.NET can read indexes that were produced by Lucene.
However, we have no tests to ensure that indexes produced by Lucene.NET 4.8.0 can be read by Lucene 4.8.0.
A way this could be done:
1. Create a command line utility that is part of the Lucene.NET build that produces a series of test index cases (at least 1 case per codec/doc values format/postings format combination). Unlike the existing compatibility tests that zip the indexes into embedded resources, we should be creating indexes based on the current build of Lucene.NET.
2. Create a Java/JUnit test project that depends on Lucene 4.8.0 and add a test per test case.
3. Add build/test tasks to azure-pipelines.yml to run the .NET utility to produce the indexes and then execute Java/JUnit tests on each target framework/OS/platform
These tests should be designed in such a way that when we upgrade to the next version of Lucene the tests are simple to upgrade as well.