new merge policy

• Key: LUCENE-672
• Type: New Feature
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Yonik Seeley
• Reporter: Yonik Seeley
• Component/s: Index
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1

Description

New merge policy developed in the course of
http://issues.apache.org/jira/browse/LUCENE-565
http://issues.apache.org/jira/secure/attachment/12340475/newMergePolicy.Sept08.patch

Comments

[Yonik Seeley]

cloned LUCENE-565 to track this separately.

[Yonik Seeley]

I just committed http://issues.apache.org/jira/secure/attachment/12340475/newMergePolicy.Sept08.patch
Thanks for the very thorough job on this patch!

[Ning Li]

A small fix named KeepDocCount0Segment.Sept15.patch is attached to LUCENE-565 (can't attach here).

In mergeSegments(...), if the doc count of a merged segment is 0, it is not added to the index (it should be properly cleaned up). Before LUCENE-672, a merged segment was always added to the index. The use of mergeSegments(...) in, e.g. addIndexes(Directory[]), assumed that behaviour. For code simplicity, this fix restores the old behaviour that a merged segment is always added to the index. This does NOT break any of the good properties of the new merge policy.

TestIndexWriterMergePolicy is slightly modified to fix a bug and to check that segments are probably cleaned up. The patch passes all the tests.

[Yonik Seeley]

Should lowerBound start off as -1 in maybeMergeSegments if we keep 0 sized segments?

[Ning Li]

> Should lowerBound start off as -1 in maybeMergeSegments if we keep 0 sized segments?

Good catch! Although the rightmost disk segment cannot be a 0-sized segment right now, it could be when NewIndexModifier is in.

Shoud I submit a new patch?

[Yonik Seeley]

No need to submit a new patch... I made the change and committed it.