[SOLR-16836] DenseVectorField to support dimensions > 1024

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.3
• Component/s: Schema and Analysis
• Labels: vector-based-search
• Assignee: Alessandro Benedetti
• Reporter: Alessandro Benedetti
• Created: 02/Jun/23 11:20
• Updated: 21/Jul/23 20:42
• Resolved: 14/Jun/23 15:31

Description

The principle objective of this PR is to make it easier for users to use Apache Solr with higher dimensions.

Right now, Lucene's arbitrary limit of 1024 is merely an ergonomics kind of limit to help the user from shooting themselves in the foot.

The underlying codec we have can read/write an arbitrary number of dimensions. CheckIndex validates the number of dimensions is greater than zero but doesn't enforce a maximum.

Potentially a warning is raised if the underline Lucene limit is exceeded.

Issue Links

• links to: GitHub Pull Request #1680