Lucene - Core
  1. Lucene - Core
  2. LUCENE-362

[PATCH] Extension to binary Fields that allows fixed byte buffer


    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core/index
    • Labels:
    • Environment:

      Operating System: All
      Platform: All


      This is a very simple patch that supports storing binary values in the index
      more efficiently. A new Field constructor accepts a length argument, allowing a
      fixed byte[] to be reused acrossed multiple calls with arguments of different
      sizes. A companion change to FieldsWriter uses this length when storing and/or
      compressing the field.

      There is one remaining case in Document. Intentionally, no direct accessor to
      the length of a binary field is provided from Document, only from Field. This
      is because Field's created by FieldReader will never have a specified length and
      this is usual case for Field's read from Document. It seems less confusing for
      most users.

      I don't believe any upward incompatibility is introduced here (e.g., from the
      possibility of getting a larger byte[] than actually holds the value from
      Document), since no such byte[] values are possible without this patch anyway.

      The compression case is still inefficient (much copying), but it is hard to see
      how Lucene can do too much better. However, the application can do the
      compression externally and pass in the reused compression-output buffer as a
      binary value (which is what I'm doing). This represents a substantialy
      allocation savings for storing large documents bodies (compressed) into the
      Lucene index.

      Two patch files are attached, both created by svn on 3/17/05.


        Chuck Williams created issue -
        Jeff Turner made changes -
        Field Original Value New Value
        issue.field.bugzillaimportkey 34066 12314512
        Chuck Williams made changes -
        Attachment FixedBufferBinaryFields.patch [ 12321328 ]
        Michael Busch made changes -
        Priority Major [ 3 ] Minor [ 4 ]
        Shai Erera made changes -
        Status Open [ 1 ] Closed [ 6 ]
        Assignee Lucene Developers [ ]
        Resolution Fixed [ 1 ]
        Mark Thomas made changes -
        Workflow jira [ 12324517 ] Default workflow, editable Closed status [ 12564104 ]
        Mark Thomas made changes -
        Workflow Default workflow, editable Closed status [ 12564104 ] jira [ 12583894 ]
        Steve Rowe made changes -
        Affects Version/s CVS Nightly - Specify date in submission [ 12310282 ]


          • Assignee:
            Chuck Williams
          • Votes:
            1 Vote for this issue
            0 Start watching this issue


            • Created: