[LUCENE-7171] IndexableField changes its IndexableFieldType when the index is re-opened for reading

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.5
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Roberto Cornacchia
• Created: 04/Apr/16 14:34
• Updated: 28/Aug/22 14:54

Description

This code:

/* Store one document into an index */
Directory index = new RAMDirectory();
IndexWriterConfig config = new IndexWriterConfig(new StandardAnalyzer());
IndexWriter w = new IndexWriter(index, config);

Document d1 = new Document();
d1.add(new StringField("isbn", "9900333X", Field.Store.YES));
w.addDocument(d1);
w.commit();
w.close();

/* inspect IndexableFieldType */
IndexableField f1 = d1.getField("isbn");
System.err.println("FieldType for " + f1.stringValue() + " : " + f1.fieldType());

/* retrieve all documents and inspect IndexableFieldType */
IndexSearcher s = new IndexSearcher(DirectoryReader.open(index));
TopDocs td = s.search(new MatchAllDocsQuery(), 1);

for (ScoreDoc sd : td.scoreDocs) {
    Document d2 = s.doc(sd.doc);
    IndexableField f2 = d2.getField("isbn");
    System.err.println("FieldType for " + f2.stringValue() + " : " + f2.fieldType());
}

Produces:

FieldType for 9900333X : stored,indexed,omitNorms,indexOptions=DOCS
FieldType for 9900333X : stored,indexed,tokenized,omitNorms,indexOptions=DOCS

The StringField field isbn is not tokenized, as correctly reported by the first output, which happens right after closing the writer.
However, it becomes tokenized when the index is re-opened with a new reader.