Lucene - Core
  1. Lucene - Core
  2. LUCENE-1924

BalancedSegmentMergePolicy, contributed from the Zoie project for realtime indexing

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.9
    • Fix Version/s: 3.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Written by Yasuhiro Matsuda for Zoie realtime indexing system used to handle high update rates to avoid large segment merges.
      Detailed write-up is at:

      http://code.google.com/p/zoie/wiki/ZoieMergePolicy

      1. BalancedSegmentMergePolicy.java
        15 kB
        John Wang
      2. LUCENE-1924.patch
        15 kB
        Michael McCandless

        Activity

        Hide
        John Wang added a comment -

        this is a stand-alone class

        Show
        John Wang added a comment - this is a stand-alone class
        Hide
        Michael McCandless added a comment -

        This looks great John, thanks. Where would you put it? contrib/misc maybe?

        Show
        Michael McCandless added a comment - This looks great John, thanks. Where would you put it? contrib/misc maybe?
        Hide
        John Wang added a comment -

        I had put them in the core package under org.apache.lucene.index.

        Because it requires access package protected classes, e.g. SegmentInfo, it needs to be in that package.

        In terms of which module, totally up to you as you would know what the best place to put it.

        A question on the side, MergePolicy is something the API suggests that is customizable, yet SegmentInfo, part of the signature is package protected. Is this something should be opened up to allow for full customization of MergePolicy amonst other things?

        Thanks

        -John

        Show
        John Wang added a comment - I had put them in the core package under org.apache.lucene.index. Because it requires access package protected classes, e.g. SegmentInfo, it needs to be in that package. In terms of which module, totally up to you as you would know what the best place to put it. A question on the side, MergePolicy is something the API suggests that is customizable, yet SegmentInfo, part of the signature is package protected. Is this something should be opened up to allow for full customization of MergePolicy amonst other things? Thanks -John
        Hide
        Michael McCandless added a comment -

        We've opened up SegmentInfo/s in 2.9.

        Show
        Michael McCandless added a comment - We've opened up SegmentInfo/s in 2.9.
        Hide
        John Wang added a comment -

        Awesome, didn't realize!

        Thanks

        -John

        Show
        John Wang added a comment - Awesome, didn't realize! Thanks -John
        Hide
        Michael McCandless added a comment -

        Trivial formatting changes, basic javadoc, and moved to contrib/misc.

        Show
        Michael McCandless added a comment - Trivial formatting changes, basic javadoc, and moved to contrib/misc.
        Hide
        Michael McCandless added a comment -

        Thanks Yasuhiro & John!

        Show
        Michael McCandless added a comment - Thanks Yasuhiro & John!
        Hide
        Per Steffensen added a comment -

        Has this been committet?

        Cannot find BalancedSegmentMergePolicy.java on neither

        http://svn.apache.org/repos/asf/lucene/dev/branches/branch_4x
        

        nor

        http://svn.apache.org/repos/asf/lucene/dev/trunk
        
        Show
        Per Steffensen added a comment - Has this been committet? Cannot find BalancedSegmentMergePolicy.java on neither http: //svn.apache.org/repos/asf/lucene/dev/branches/branch_4x nor http: //svn.apache.org/repos/asf/lucene/dev/trunk
        Hide
        Michael McCandless added a comment -

        It was committed, but there were issues (I believe a test was failing at the time), but nobody had the energy/time to fix it ... so we removed it at some point.

        Show
        Michael McCandless added a comment - It was committed, but there were issues (I believe a test was failing at the time), but nobody had the energy/time to fix it ... so we removed it at some point.
        Hide
        Per Steffensen added a comment -

        Ok, thanks!

        Show
        Per Steffensen added a comment - Ok, thanks!

          People

          • Assignee:
            Michael McCandless
            Reporter:
            John Wang
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development