Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5638 Default Attributes are expensive
  3. LUCENE-5640

Cleanup & deprecate Token class / Improve default AttributeFactory to no longer use reflection



    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.9, 6.0
    • modules/analysis
    • None
    • New


      We should remove code duplication in the Token class:

      • copy constructors
      • reinit() shit
      • non-default clone()

      This is too bugy. Most of the methods can be simply removed.

      This issue will also factor out the basic attributes to a separate implementation class (without the Token extra stuff). Token then just extends this class (in the tokenattributes package) and adds the additional stuff not needed for an Attribute. Token itsself will get deprecated.

      Also part of the slowdown in the parent issue is caused by ineffective DefaultAttributeFactory, which uses reflection to instantiate new attributes. As we are on Java 7 now, we can use MethodHandle to do this. MethodHandle does access checks only once on creating the factory or when the attribute is seen first. Later invocations are done without any result type conversions and parameter conversions as a statically linked constructor call. This greatly improves performance with Java 8, Java 7 is approx as fast, unless its completely static.


        1. LUCENE-5640-4x.patch
          141 kB
          Uwe Schindler
        2. LUCENE-5640.patch
          14 kB
          Robert Muir
        3. LUCENE-5640.patch
          102 kB
          Uwe Schindler
        4. LUCENE-5640.patch
          106 kB
          Uwe Schindler
        5. LUCENE-5640.patch
          106 kB
          Uwe Schindler
        6. LUCENE-5640.patch
          109 kB
          Uwe Schindler
        7. LUCENE-5640.patch
          114 kB
          Uwe Schindler
        8. LUCENE-5640.patch
          131 kB
          Uwe Schindler



            uschindler Uwe Schindler
            uschindler Uwe Schindler
            0 Vote for this issue
            2 Start watching this issue