[SOLR-1876] Convert all tokenstreams and tests to use CharTermAttribute

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1
• Fix Version/s: 3.1, 4.0-ALPHA
• Component/s: Schema and Analysis
• Assignee: Robert Muir
• Reporter: Robert Muir
• Created: 10/Apr/10 16:01
• Updated: 30/Mar/11 15:45
• Resolved: 11/Apr/10 10:26

Description

See the improvements in LUCENE-2302.

TermAttribute has been deprecated for flexible indexing, as terms can really be anything, as long as they can
be serialized to byte[].

For character-terms, a CharTermAttribute has been created, with a more friendly API. Additionally this attribute
implements the CharSequence and Appendable interfaces.

We should convert all Solr tokenstreams to use this new attribute.

Issue Links

• is part of: LUCENE-2372 Replace deprecated TermAttribute by new CharTermAttribute
• is part of: LUCENE-2389 Enforce TokenStream impl / Analyzer finalness by an assertion