Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7943

Plane.findArcDistancePoints() sometimes throws assertion failures even when plane explicitly normalized

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 6.7, 7.1, 8.0
    • None
    • None
    • New

    Description

      The following assertion sometimes fails even when the plane has been explicitly normalized:

          assert Math.abs(x*x + y*y + z*z - 1.0) < MINIMUM_RESOLUTION_SQUARED : "Plane needs to be normalized";
      

      I can find nothing wrong with the assertion check, but the numerical accuracy is 1e-16, which just isn't high enough to support MINIMUM_RESOLUTION_SQUARED accuracy (1e-24).

      Attachments

        Activity

          People

            kwright@metacarta.com Karl Wright
            kwright@metacarta.com Karl Wright
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: