[LUCENE-7591] Let DatasetSplitter approximate no. of class values by no. of terms

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.0
• Component/s: modules/classification
• Lucene Fields: New
• Assignee: Tommaso Teofili
• Reporter: Tommaso Teofili
• Created: 12/Dec/16 08:59
• Updated: 28/Aug/22 15:07
• Resolved: 12/Dec/16 09:00

Description

Currently DatasetSplitter throws an exception if it's not possible to find SortedDocValues or SortedSetDocValues on the class field as it wouldn't be possible to correctly split the indexes in a balanced way.
As a fallback we could instead use the no. of terms per leaf reader as an approximate count (upper bound) to the no. of classes.