Solr
  1. Solr
  2. SOLR-935

DataImportHandler: Add logging to record failure to acquire lock by DataImporter for a given request

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.9, 5.0
    • Labels:
      None
    • Environment:

      Java 6, Tomcat 6.0.18

      Description

      There is a possibility of 2 threads to be in DataImporter:runCmd, until before importLock.tryLock() method and then depending on the scheduling - one of them is allowed to pass through from then .

      We need to log the failure of the other as to unable to start because of the failure to acquire the mutex, to distinguish between successful start of import and failure to do so.

      1. SOLR-935.patch
        0.7 kB
        Noble Paul
      2. SOLR-935.patch
        6 kB
        Karthik K
      3. SOLR-935.patch
        0.7 kB
        Karthik K

        Issue Links

          Activity

          Hide
          Shalin Shekhar Mangar added a comment -

          We can add this piece of logging but what benefit will that provide? Will it scare users when they see something like "Mutex Acquisition failure" in the logs?

          Show
          Shalin Shekhar Mangar added a comment - We can add this piece of logging but what benefit will that provide? Will it scare users when they see something like "Mutex Acquisition failure" in the logs?
          Hide
          Karthik K added a comment -

          What it really means is that - after the full/delta import request is submitted - it did not launch at all in the server side, but rather failed quietly.

          It is important from a debugging perspective that we see something like a "warning: data import did not launch because of mutex acquisition failure. Please retry" .

          Show
          Karthik K added a comment - What it really means is that - after the full/delta import request is submitted - it did not launch at all in the server side, but rather failed quietly. It is important from a debugging perspective that we see something like a "warning: data import did not launch because of mutex acquisition failure. Please retry" .
          Hide
          Karthik K added a comment -

          Increasing priority since in a workflow scenario with more than 2 delta-import requests received at the server - we need some way to track in the debug log that one of them actually failed and need to be tried again.

          Show
          Karthik K added a comment - Increasing priority since in a workflow scenario with more than 2 delta-import requests received at the server - we need some way to track in the debug log that one of them actually failed and need to be tried again.
          Hide
          Karthik K added a comment -

          New Patch submitted for workflow modification to track failures better.

          • Logging is removed in favor of more granular notification.

          Context:
          New field called EventStatus added to Context ( default value - SUCCESS)

          During failure - another event is launched , with the EventStatus field in the Context set to Failure.

          Makes it easy for a plugin to keep track of failures when the delta import did not launch at all (due to synch. issues), compared to log based debugging.

          This patch is independent of SOLR-972, but would be extremely efficient if SOLR-972 is applied though.

          Show
          Karthik K added a comment - New Patch submitted for workflow modification to track failures better. Logging is removed in favor of more granular notification. Context: New field called EventStatus added to Context ( default value - SUCCESS) During failure - another event is launched , with the EventStatus field in the Context set to Failure. Makes it easy for a plugin to keep track of failures when the delta import did not launch at all (due to synch. issues), compared to log based debugging. This patch is independent of SOLR-972 , but would be extremely efficient if SOLR-972 is applied though.
          Hide
          Karthik K added a comment -

          For high performance of this patch (related to logging of failure) - applying SOLR-972 is highly recommended.

          Show
          Karthik K added a comment - For high performance of this patch (related to logging of failure) - applying SOLR-972 is highly recommended.
          Hide
          Shalin Shekhar Mangar added a comment -

          The logging is fine. But I am not sure of the utility of the event listener. Why would one configure an import to be run so frequently that it can trigger the failure to acquire lock? What useful action can one take in this scenario?

          Show
          Shalin Shekhar Mangar added a comment - The logging is fine. But I am not sure of the utility of the event listener. Why would one configure an import to be run so frequently that it can trigger the failure to acquire lock? What useful action can one take in this scenario?
          Hide
          Karthik K added a comment -

          Makes it easy for a plugin to keep track of failures when the delta import did not launch at all

          Otherwise those frequent imports ( due to a code / workflow error ) would go unnoticed.

          Show
          Karthik K added a comment - Makes it easy for a plugin to keep track of failures when the delta import did not launch at all Otherwise those frequent imports ( due to a code / workflow error ) would go unnoticed.
          Hide
          Noble Paul added a comment -

          the partial fix. It logs a warning if it fails to acquire a lock

          I am still not convinced of the EventListener part

          Show
          Noble Paul added a comment - the partial fix. It logs a warning if it fails to acquire a lock I am still not convinced of the EventListener part
          Hide
          Noble Paul added a comment -

          committed revision : 776965

          Show
          Noble Paul added a comment - committed revision : 776965
          Hide
          Noble Paul added a comment -

          I feel that the original concern is addressed. I am pushing it to 1.5 so that we can have a detailed look at this later

          Show
          Noble Paul added a comment - I feel that the original concern is addressed. I am pushing it to 1.5 so that we can have a detailed look at this later
          Hide
          Hoss Man added a comment -

          Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

          http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

          Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

          A unique token for finding these 240 issues in the future: hossversioncleanup20100527

          Show
          Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
          Hide
          Robert Muir added a comment -

          Bulk move 3.2 -> 3.3

          Show
          Robert Muir added a comment - Bulk move 3.2 -> 3.3
          Hide
          Robert Muir added a comment -

          3.4 -> 3.5

          Show
          Robert Muir added a comment - 3.4 -> 3.5
          Hide
          Hoss Man added a comment -

          bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

          Show
          Hoss Man added a comment - bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
          Hide
          Robert Muir added a comment -

          rmuir20120906-bulk-40-change

          Show
          Robert Muir added a comment - rmuir20120906-bulk-40-change
          Hide
          Robert Muir added a comment -

          moving all 4.0 issues not touched in a month to 4.1

          Show
          Robert Muir added a comment - moving all 4.0 issues not touched in a month to 4.1
          Hide
          Steve Rowe added a comment -

          Bulk move 4.4 issues to 4.5 and 5.0

          Show
          Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
          Hide
          Uwe Schindler added a comment -

          Move issue to Solr 4.9.

          Show
          Uwe Schindler added a comment - Move issue to Solr 4.9.

            People

            • Assignee:
              Noble Paul
              Reporter:
              Karthik K
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development