Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9215

replace checkJavaDocs.py with doclet


    • Task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 9.0
    • None
    • None
    • New


      The current checker runs regular expressions against html, and it breaks when newer java change html output. This is not particularly fun to fix: see LUCENE-9213

      Java releases often now, and when i compared generated html of a simple class across 11,12,13 it surprised me how much changes. So I think we want to avoid parsing their HTML.

      Javadoc Xdoclint feature has a "missing checker": but it is black/white. Either everything is fully documented or its not. And while you can enable/disable doclint checks per-package, this also seems black/white (either all checks or no checks at all).

      On the other hand the python checker is able to check per-package at different granularities (package, class, method). It makes it possible to iteratively improve the situation.

      With doclet api we could implement checks in pure java, for example to match checkJavaDocs.py logic:

        private void checkComment(Element element) {
          var tree = docTrees.getDocCommentTree(element);
          if (tree == null) {
            error(element, "javadocs are missing");
          } else {
            var normalized = tree.getFirstSentence().get(0).toString()
                             .replace('\u00A0', ' ')
            if (normalized.isEmpty()) {
              error(element, "blank javadoc comment");
            } else if (normalized.startsWith("licensed to the apache software foundation") ||
                       normalized.startsWith("copyright 2004 the apache software foundation")) {
              error(element, "comment is really a license");

      If there are problems then they just appear as errors from the output of javadoc like usual:

      javadoc: error - org.apache.lucene.nodoc (package): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanNearQuery.java:190: error - SpanNearWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanContainingQuery.java:54: error - SpanContainingWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanWithinQuery.java:55: error - SpanWithinWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanTermQuery.java:94: error - SpanTermWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanNotQuery.java:109: error - SpanNotWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanOrQuery.java:139: error - SpanOrWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/spans/SpanPositionCheckQuery.java:77: error - SpanPositionCheckWeight (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/MultiCollectorManager.java:61: error - Collectors (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/search/MultiCollectorManager.java:89: error - LeafCollectors (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/util/PagedBytes.java:353: error - PagedBytesDataOutput (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/util/PagedBytes.java:285: error - PagedBytesDataInput (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/nodoc/EmptyDoc.java:22: error - EmptyDoc (class): javadocs are missing
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/nodoc/LicenseDoc.java:36: error - LicenseDoc (class): comment is really a license
      /home/rmuir/workspace/lucene-solr/lucene/core/src/java/org/apache/lucene/nodoc/NoDoc.java:19: error - NoDoc (class): javadocs are missing
      FAILURE: Build failed with an exception.
      * What went wrong:
      Execution failed for task ':lucene:core:javadoc'.
      > Javadoc generation failed. Generated Javadoc options file (useful for troubleshooting): '/home/rmuir/workspace/lucene-solr/lucene/core/build/tmp/javadoc/javadoc.options'


        1. LUCENE-9215_prototype.patch
          4 kB
          Robert Muir
        2. overrides.patch
          6 kB
          Dawid Weiss

        Issue Links



              rcmuir Robert Muir
              rcmuir Robert Muir
              0 Vote for this issue
              3 Start watching this issue



                Time Tracking

                  Original Estimate - Not Specified
                  Not Specified
                  Remaining Estimate - 0h
                  Time Spent - 5.5h