Solr
  1. Solr
  2. SOLR-6834

checkIntegrityAtMerge needs removed from example configs & a warning should be logged if used

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      from the dev@lucene list...

      Subject: Re: Performance hit of Solr checkIntegrityAtMerge

      There are two costs: cpu and i/o.

      The cpu cost is not much anyway but can be made basically trivial by
      using java 8.
      The i/o cost is because the check is not done with any i/o locality to
      the data being merged. so it could be a perf hit for an extremely
      large merge.

      In 5.0 the option is removed: we reworked this computation in merging
      to always have locality and so on, the checking always happens.

      ...but on the 5x branch, the checkIntegrityAtMerge setting (and comments) still exist in the example configs, and the SolrIndexConfig code still parses it (but does nothing with it since IWC no longer accepts it as an option)

      todo..

      • remove setting form example configs (runk & 5x)
      • update code to check if the setting is used & log a warning that it's now ignored
        • backport this to 5x
      • update code (trunk only) to completely remove parsing of this setting

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          patch i worked up earlier ... didn't test or precommit yet, but i'll look at it with fresh eyes in the morning.

          Show
          Hoss Man added a comment - patch i worked up earlier ... didn't test or precommit yet, but i'll look at it with fresh eyes in the morning.
          Hide
          Hoss Man added a comment -

          related issue: LUCENE-5969 is where IWC stoped having a setCheckIntegrityAtMerge

          Show
          Hoss Man added a comment - related issue: LUCENE-5969 is where IWC stoped having a setCheckIntegrityAtMerge
          Hide
          ASF subversion and git services added a comment -

          Commit 1644707 from hossman@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1644707 ]

          SOLR-6834: Warn if checkIntegrityAtMerge is configured

          Show
          ASF subversion and git services added a comment - Commit 1644707 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1644707 ] SOLR-6834 : Warn if checkIntegrityAtMerge is configured
          Hide
          ASF subversion and git services added a comment -

          Commit 1644739 from hossman@apache.org in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1644739 ]

          SOLR-6834: Warn if checkIntegrityAtMerge is configured (merge r1644707)

          Show
          ASF subversion and git services added a comment - Commit 1644739 from hossman@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1644739 ] SOLR-6834 : Warn if checkIntegrityAtMerge is configured (merge r1644707)
          Hide
          ASF subversion and git services added a comment -

          Commit 1644759 from hossman@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1644759 ]

          SOLR-6834: hard fail if checkIntegrityAtMerge is used in 6.0 (trunk only, not for backport)

          Show
          ASF subversion and git services added a comment - Commit 1644759 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1644759 ] SOLR-6834 : hard fail if checkIntegrityAtMerge is used in 6.0 (trunk only, not for backport)
          Hide
          Anshum Gupta added a comment -

          Bulk close after 5.0 release.

          Show
          Anshum Gupta added a comment - Bulk close after 5.0 release.

            People

            • Assignee:
              Hoss Man
              Reporter:
              Hoss Man
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development