[LUCENE-2985] Build SegmentCodecs incrementally for consistent codecIDs during indexing

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0-ALPHA, CSF branch
• Fix Version/s: 4.0-ALPHA, CSF branch
• Component/s: core/codecs, core/index
• Lucene Fields: New, Patch Available
• Assignee: Simon Willnauer
• Reporter: Simon Willnauer
• Created: 23/Mar/11 12:25
• Updated: 28/Aug/22 12:43
• Resolved: 25/Mar/11 16:24

Description

currently we build the SegementCodecs during flush which is fine as long as no codec needs to know which fields it should handle. This will change with DocValues or when we expose StoredFields / TermVectors via Codec (see LUCENE-2621 or LUCENE-2935). The other downside it that we don't have a consistent view of which codec belongs to which field during indexing and all FieldInfo instances are unassigned (set to -1). Instead we should build the SegmentCodecs incrementally as fields come in so no matter when a codec needs to be selected to process a document / field we have the right codec ID assigned.

Issue Links

• blocks: LUCENE-2621 Extend Codec to handle also stored fields and term vectors
• relates to: LUCENE-2935 Let Codec consume entire document