[LUCENE-4176] Can not produce proper collation key for ICUCollatedTermAttributeImp

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.0
• Fix Version/s: 4.0-BETA, 6.0
• Component/s: modules/queryparser
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Nattapong Sirilappanich
• Created: 28/Jun/12 11:51
• Updated: 09/May/16 18:32
• Resolved: 28/Jun/12 13:30

Description

org.apache.lucene.collation.tokenattributes.ICUCollatedTermAttributeImpl return a hash of collation key's byte.
The given hash value produce incorrect comparison result.
The source code below return 1 for Lucene 3.6.
The code here return 0.
Code to reproduce:

IndexWriter writer = new IndexWriter(ramDir, conf);
Document doc = new Document();
FieldType fieldType = new FieldType();
fieldType.setIndexed(true);
fieldType.setStored(true);
Field field = new Field("content","เข", fieldType);
doc.add(field);
writer.addDocument(doc);
writer.close();
IndexSearcher is = new IndexSearcher(DirectoryReader.open(ramDir));
QueryParser qp = new AnalyzingQueryParser(Version.LUCENE_50,"content", analyzer);

ScoreDoc[] result = is.search(qp.parse("[\u0e01 TO \u0e03]"), null,1000).scoreDocs;
System.out.println(result.length);