ManifoldCF
  1. ManifoldCF
  2. CONNECTORS-299

Crawling using Postgresql 9.1 hangs occasionally for a while

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: ManifoldCF 0.4
    • Fix Version/s: ManifoldCF 0.4
    • Component/s: Framework core
    • Labels:
      None

      Description

      The hang takes place because of this postgresql error:

      ERROR 2011-11-28 14:46:23,959 (Worker thread '25') - Worker thread aborting and restarting due to database connection reset: Database exception: Exception doing query: ERROR: could not serialize access due to read/write dependencies among transactions
      Detail: Reason code: Canceled on identification as a pivot, during commit attempt.
      Hint: The transaction might succeed if retried.
      org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database exception: Exception doing query: ERROR: could not serialize access due to read/write dependencies among transactions
      Detail: Reason code: Canceled on identification as a pivot, during commit attempt.
      Hint: The transaction might succeed if retried.
      at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:608)
      at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.commitCurrentTransaction(DBInterfacePostgreSQL.java:1141)
      at org.apache.manifoldcf.core.database.Database.endTransaction(Database.java:321)
      at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.endTransaction(DBInterfacePostgreSQL.java:1112)
      at org.apache.manifoldcf.crawler.jobs.JobManager.finishDocuments(JobManager.java:4072)
      at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:567)
      Caused by: org.postgresql.util.PSQLException: ERROR: could not serialize access due to read/write dependencies among transactions

      This is happening during the commit phase of the transaction, at a point where none of the transaction blocks in ManifoldCF know to look for it. It is therefore likely that all transaction blocks throughout ManifoldCF that deal with deadlock will need to be changed.

        Activity

        Karl Wright made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Karl Wright made changes -
        Field Original Value New Value
        Description The hang takes place because of this postgresql error:

        ---- message: ERROR: could not serialize access due to read/write dependencies among transactions
           Detail: Reason code: Canceled on identification as a pivot, during conflict in checking.
           Hint: The transaction might succeed if retried., ----- state: 40001

        The state code 40001 should be interpreted as a "deadlock" so that the transaction is retried.
        The hang takes place because of this postgresql error:

        ERROR 2011-11-28 14:46:23,959 (Worker thread '25') - Worker thread aborting and restarting due to database connection reset: Database exception: Exception doing query: ERROR: could not serialize access due to read/write dependencies among transactions
         Detail: Reason code: Canceled on identification as a pivot, during commit attempt.
         Hint: The transaction might succeed if retried.
        org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database exception: Exception doing query: ERROR: could not serialize access due to read/write dependencies among transactions
         Detail: Reason code: Canceled on identification as a pivot, during commit attempt.
         Hint: The transaction might succeed if retried.
               at org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:608)
               at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.commitCurrentTransaction(DBInterfacePostgreSQL.java:1141)
               at org.apache.manifoldcf.core.database.Database.endTransaction(Database.java:321)
               at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.endTransaction(DBInterfacePostgreSQL.java:1112)
               at org.apache.manifoldcf.crawler.jobs.JobManager.finishDocuments(JobManager.java:4072)
               at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:567)
        Caused by: org.postgresql.util.PSQLException: ERROR: could not serialize access due to read/write dependencies among transactions

        This is happening during the commit phase of the transaction, at a point where none of the transaction blocks in ManifoldCF know to look for it. It is therefore likely that all transaction blocks throughout ManifoldCF that deal with deadlock will need to be changed.
        Karl Wright created issue -

          People

          • Assignee:
            Karl Wright
            Reporter:
            Karl Wright
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development