Solr
  1. Solr
  2. SOLR-4934

Prevent runtime failure if users use initargs "useCompoundFile" setting on LogMergePolicy or TieredMergePolicy

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.4, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      • LUCENE-5038 eliminated setUseCompoundFile(boolean) from the built in MergePolicies
      • existing users may have configs that use <mergePolicy> init args to try and call that setter
      • we already do some explicit checks for these MergePolices in SolrIndexConfig to deal with legacy syntax
      • update the existing logic to remove "useCompoundFile" from the MergePolicy initArgs for these known policies if found, and log a warning.

      (NOTE: i don't want to arbitrarily remove "useCompoundFile" from the initArgs regardless of class in case someone has a custom MergePolicy that implements that logic – that would suck)

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          Committed revision 1494348.

          The fix itself was fairly small, and the bulk of the change was svn copying of test configs so i just went ahead and committed to trunk instead of attaching a patch.

          If there are no objections, i'll backport to 4x later tonight or early tommorow.

          Show
          Hoss Man added a comment - Committed revision 1494348. The fix itself was fairly small, and the bulk of the change was svn copying of test configs so i just went ahead and committed to trunk instead of attaching a patch. If there are no objections, i'll backport to 4x later tonight or early tommorow.
          Hide
          Shawn Heisey added a comment -

          I was getting ready to file an issue, glad I found this before doing so. The only thing I knew was that LUCENE-5038 had caused Solr to make compound files and the useCompoundFile setting under indexConfig that I found in the branch_4x example wasn't turning it off.

          A connected discussion, for which I can file an issue if necessary: Assuming there are plenty of file descriptors available, will a user get better performance from compound files or separate files? Is it dependent on other factors like filesystem choice, or is one a clear winner? The outcome of that discussion should decide what Solr's default is when no related config options are used.

          Show
          Shawn Heisey added a comment - I was getting ready to file an issue, glad I found this before doing so. The only thing I knew was that LUCENE-5038 had caused Solr to make compound files and the useCompoundFile setting under indexConfig that I found in the branch_4x example wasn't turning it off. A connected discussion, for which I can file an issue if necessary: Assuming there are plenty of file descriptors available, will a user get better performance from compound files or separate files? Is it dependent on other factors like filesystem choice, or is one a clear winner? The outcome of that discussion should decide what Solr's default is when no related config options are used.
          Hide
          Hoss Man added a comment - - edited

          The only thing I knew was that LUCENE-5038 had caused Solr to make compound files and the useCompoundFile setting under indexConfig that I found in the branch_4x example wasn't turning it off.

          Oh ... hmmm, yeah ... i hadn't noticed that. definitely a bug there. I've opened SOLR-4941 to track that, and we'll leave this issue specifically about the broken initargs config option.

          EDIT: fixed issue number

          Show
          Hoss Man added a comment - - edited The only thing I knew was that LUCENE-5038 had caused Solr to make compound files and the useCompoundFile setting under indexConfig that I found in the branch_4x example wasn't turning it off. Oh ... hmmm, yeah ... i hadn't noticed that. definitely a bug there. I've opened SOLR-4941 to track that, and we'll leave this issue specifically about the broken initargs config option. EDIT: fixed issue number
          Hide
          Hoss Man added a comment -

          merged r1494348 -> 4x as r1494696

          Show
          Hoss Man added a comment - merged r1494348 -> 4x as r1494696
          Hide
          Uwe Schindler added a comment -

          Assuming there are plenty of file descriptors available, will a user get better performance from compound files or separate files?

          Searching on the index will have no negative impact. IndexInputSlicer returns optimized indexinputs that remove the whole file offset stuff. Indexing speed is identical, too, but merging (done in background) is more expensive.

          Show
          Uwe Schindler added a comment - Assuming there are plenty of file descriptors available, will a user get better performance from compound files or separate files? Searching on the index will have no negative impact. IndexInputSlicer returns optimized indexinputs that remove the whole file offset stuff. Indexing speed is identical, too, but merging (done in background) is more expensive.
          Hide
          Steve Rowe added a comment -

          Bulk close resolved 4.4 issues

          Show
          Steve Rowe added a comment - Bulk close resolved 4.4 issues

            People

            • Assignee:
              Hoss Man
              Reporter:
              Hoss Man
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development