[LUCENE-5638] Default Attributes are expensive

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.9, 6.0
• Component/s: modules/analysis
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Robert Muir
• Created: 02/May/14 16:34
• Updated: 09/May/16 18:35
• Resolved: 04/May/14 15:23

Description

Changes like LUCENE-5634 make it clear that the default AttributeFactory stuff has a very high cost: weakmaps/reflection/etc.

Additionally I think clearAttributes() is more expensive than it should be: it has to traverse a linked-list, calling clear() per token.

Operations like cloning (save/restoreState) have a high cost tll.

Maybe we can have a better Default? In other words, rename DEFAULT_ATTRIBUTE_FACTORY to REFLECTION_ATTRIBUTE_FACTORY, and instead have a faster default factory that just has one AttributeImpl with the "core ones" that 95% of users are dealing with (TOKEN_ATTRIBUTE_FACTORY?): anything outside of that falls back to reflection.

Sub-Tasks

• Fix implementation of PositionLengthAttribute in Token.java: Resolved
• Cleanup & deprecate Token class / Improve default AttributeFactory to no longer use reflection: Resolved
• Randomize attributefactory in MockTokenizer, TestRandomChains, etc.: Resolved