[LUCENE-3504] DocValues: deref/sorted bytes types shouldn't return empty byte[] when doc didn't have a value

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 4.0-ALPHA
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 10/Oct/11 22:00
• Updated: 28/Aug/22 12:59
• Resolved: 11/Oct/11 19:19

Description

I'm looking at making a FieldComparator that uses DV's SortedSource to
sort by string field (ie just like TermOrdValComparator, except using
DV instead of FieldCache). We already have comparators for DV int and
float DV fields.

But one thing I noticed is we can't detect documents that didn't have
any value indexed vs documents that had empty byte[] indexed.

This is easy to fix (and we used to do this), because these types are
deref'd (ie, each doc stores an address, and then separately looks up
the byte[] at that address), we can reserve ord/address 0 to mean "doc
didn't have the field". Then we should return null when you retrieve
the BytesRef value for that field.