Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14778

Disabling UpdateLog leads to silently lost updates

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 8.6.1
    • Fix Version/s: None
    • Component/s: SolrCloud, update
    • Labels:
      None

      Description

      Solr currently "supports" disabling the UpdateLog, though it is "required" for NRT replicas (per the docs). However, when the update log is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd or SplitShardCmd), updates areĀ lost silently. While most users will likely never consider disabling the updateLog, it seems pertinent to provide a better support option.

      Options as discussed in ASF Slack:

      1. No longer support disabling the updateLog as it is considered an integral feature in SolrCloud. This might be undesirable for use cases where some data loss is acceptable and the updateLog takes up too much space.
      2. Improve Solr documentation to explicitly outline the risks of disabling the updateLog.
      3. Add logging to indicate when an update is swallowed in this state.
      4. My preferred option: Support disabling the updateLog by providing additional replica states besides BUFFERING, so that there is no data loss when updateLog is disabled and replica goes offline for an operation like split. Some ideas:
        1. REJECTING: Fail updates so that the client can retry again once the operation is complete.
        2. BLOCKING: Stall update until operation is complete, and then execute update.

      Feedback is welcome; once we establish a path forward I'd be happy to pick it up. If others are interested I can document my findings as well.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              megancarey Megan Carey
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: