ManifoldCF
  1. ManifoldCF
  2. CONNECTORS-453

ManifoldCF running with Derby 10.8.1.1 has problems pausing and aborting jobs

    Details

      Description

      Since upgrading to Derby 10.8.x.x, tt takes minutes to crawl just 20 documents. Clearly the Derby contention/locking bugs are back with a vengeance in 10.8.x.x. Either we use 10.7.x.x or we get the Derby team to look at them again.

      In the interim, maybe it is time to use hsqldb as the default embedded database for the single-process example instead of Derby.

        Activity

        Hide
        Karl Wright added a comment -

        I see stalls only at the very beginning of a crawl. Long crawls with lots of documents don't appear to stall, however. Still trying to figure out if this is an actual problem or something more innocuous.

        Show
        Karl Wright added a comment - I see stalls only at the very beginning of a crawl. Long crawls with lots of documents don't appear to stall, however. Still trying to figure out if this is an actual problem or something more innocuous.
        Hide
        Karl Wright added a comment -

        Clicking pause during the job run yields the following to be displayed in the UI:

        A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID :

        {147028, X}

        , APP, UPDATE jobs SET status=? WHERE id=? Granted XID :

        {146703, S}

        ,

        {146941, S} Lock : ROW, JOBQUEUE, (481,10) Waiting XID : {146941, S}

        , APP, SELECT jobid,CAST(COUNT(dochash) AS bigint) AS doccount FROM jobqueue t1 WHERE EXISTS(SELECT 'x' FROM jobs t0 WHERE t0.id=t1.jobid AND id=?) GROUP BY jobid Granted XID :

        {146612, X}

        Lock : ROW, HOPCOUNT, (1734,27) Waiting XID :

        {146612, S}

        , APP, SELECT parentidhash,linktype,distance FROM hopcount WHERE jobid=? AND parentidhash IN (?,?,?,?,?,?,?,?,?,?) AND linktype=? Granted XID :

        {146666, X}

        Lock : ROW, JOBS, (1,7) Waiting XID :

        {146666, S}

        , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 147028.

        Show
        Karl Wright added a comment - Clicking pause during the job run yields the following to be displayed in the UI: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID : {147028, X} , APP, UPDATE jobs SET status=? WHERE id=? Granted XID : {146703, S} , {146941, S} Lock : ROW, JOBQUEUE, (481,10) Waiting XID : {146941, S} , APP, SELECT jobid,CAST(COUNT(dochash) AS bigint) AS doccount FROM jobqueue t1 WHERE EXISTS(SELECT 'x' FROM jobs t0 WHERE t0.id=t1.jobid AND id=?) GROUP BY jobid Granted XID : {146612, X} Lock : ROW, HOPCOUNT, (1734,27) Waiting XID : {146612, S} , APP, SELECT parentidhash,linktype,distance FROM hopcount WHERE jobid=? AND parentidhash IN (?,?,?,?,?,?,?,?,?,?) AND linktype=? Granted XID : {146666, X} Lock : ROW, JOBS, (1,7) Waiting XID : {146666, S} , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 147028.
        Hide
        Karl Wright added a comment -

        Here's another example:

        Error!
        A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID :

        {157800, X}

        , APP, UPDATE jobs SET status=? WHERE id=? Granted XID :

        {157521, S}

        ,

        {157653, S} Lock : ROW, JOBQUEUE, (503,86) Waiting XID : {157653, S}

        , APP, SELECT t0.id,t0.jobid,t0.dochash,t0.docid,t0.status,t0.failtime,t0.failcount,t0.priorityset FROM jobqueue t0 WHERE t0.status IN (?,?) AND t0.checkaction=? AND t0.checktime<=? AND EXISTS(SELECT 'x' FROM jobs t1 WHERE t1.status IN (?,?) AND t1.id=t0.jobid AND t1.priority=?) AND NOT EXISTS(SELECT 'x' FROM jobqueue t2 WHERE t2.dochash=t0.dochash AND t2.status IN (?,?,?,?,?,?) AND t2.jobid!=t0.jobid) AND NOT EXISTS(SELECT 'x' FROM prereqevents t3,events t4 WHERE t0.id=t3.owner AND t3.eventname=t4.name) ORDER BY t0.docpriority ASC,t0.status ASC,t0.checkaction ASC,t0.checktime ASC FETCH NEXT 120 ROWS ONLY Granted XID :

        {157557, X}

        Lock : ROW, JOBS, (1,7) Waiting XID :

        {157557, S}

        , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 157800.

        Show
        Karl Wright added a comment - Here's another example: Error! A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, JOBS, (1,7) Waiting XID : {157800, X} , APP, UPDATE jobs SET status=? WHERE id=? Granted XID : {157521, S} , {157653, S} Lock : ROW, JOBQUEUE, (503,86) Waiting XID : {157653, S} , APP, SELECT t0.id,t0.jobid,t0.dochash,t0.docid,t0.status,t0.failtime,t0.failcount,t0.priorityset FROM jobqueue t0 WHERE t0.status IN (?,?) AND t0.checkaction=? AND t0.checktime<=? AND EXISTS(SELECT 'x' FROM jobs t1 WHERE t1.status IN (?,?) AND t1.id=t0.jobid AND t1.priority=?) AND NOT EXISTS(SELECT 'x' FROM jobqueue t2 WHERE t2.dochash=t0.dochash AND t2.status IN (?,?,?,?,?,?) AND t2.jobid!=t0.jobid) AND NOT EXISTS(SELECT 'x' FROM prereqevents t3,events t4 WHERE t0.id=t3.owner AND t3.eventname=t4.name) ORDER BY t0.docpriority ASC,t0.status ASC,t0.checkaction ASC,t0.checktime ASC FETCH NEXT 120 ROWS ONLY Granted XID : {157557, X} Lock : ROW, JOBS, (1,7) Waiting XID : {157557, S} , APP, INSERT INTO hopcount (deathmark,parentidhash,id,distance,jobid,linktype) VALUES (?,?,?,?,?,?) . The selected victim is XID : 157800.
        Hide
        Karl Wright added a comment -

        r1331102

        Show
        Karl Wright added a comment - r1331102
        Hide
        Karl Wright added a comment -

        Reopening for inclusion in 0.5.1

        Show
        Karl Wright added a comment - Reopening for inclusion in 0.5.1
        Hide
        Karl Wright added a comment -

        r1337457 (release branch)

        Show
        Karl Wright added a comment - r1337457 (release branch)

          People

          • Assignee:
            Karl Wright
            Reporter:
            Karl Wright
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development