Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-1118

ConcurrentLockManager issue: lock conversion and the same lock request from same transaction generates the abortion of that transaction

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • None
    • *DB - AsterixDB, TX - Transactions
    • None

    Description

      There is an issue that is related to a DELETE query that can be transformed as a index-only phase 2 plan (only reducing the number of select plan) in my branch. In this plan, during a secondary index search, a tryLock on PK will be tried. If it succeeds, the verification (SELECT) after the primary index can be skipped. If it fails, the verification will be applied.

      So, for my branch, the issue is that, during a secondary index search (name = "REDIN.com"), a S lock on PK will be placed. After that, during the primary index search, another S lock on the same PK will be placed. So far, so good. After that, to delete that tuple, DELETE operator tries to get a X lock on the same PK and it fails. The current lock manager doesn't have a mechanism to convert S lock to X lock when the request comes from the same transaction. Young-Seok and I also tried to change lock types from S to X in the all places of the search (secondary and primary). Still, it generates an error.

      To summarize,

      1) a transaction has S lock. It requests X lock on the same PK -> transaction abort
      2) a transaction has X lock. It requests X lock on the same PK -> transaction abort

      In the current master, this problem doesn't appear since a secondary index search doesn't place any lock. And the primary index search is instant try lock. Thus, when DELETE operator tries to get a X lock, it succeeds.

      ///////////////////
      drop dataverse STBench if exists;
      create dataverse STBench;
      use dataverse STBench;
      create type SimpleGeoPlaceType as closed

      { coordinates: point, /* geometry -> coordinates (first -> should be x, second -> should be y) */ id: int64, name: string, /* properties -> name */ tags: string, /* properties -> tags (arrary of string) */ categories: string, /* properties -> classifiers -> category, type, subcategory (will be concatenated as a string) */ phone: string /* properties -> phone */ } create dataset SimpleGeoPlace (SimpleGeoPlaceType) primary key id; create index btreeName on SimpleGeoPlace(name); insert into dataset SimpleGeoPlace( { "coordinates": point("0.5,0.5"), "id": 10100000, "name": "REIDIN.com", "tags": "real-estate property emerging-markets business-intelligence china india russia turkey brazil dubai uae", "categories": "Professional Services Computer Services", "phone": "" }

      );

      delete $x from dataset SimpleGeoPlace
      where $x.name = "REIDIN.com"

      ////////////

      Error message:
      push Job 844424930131968
      push Resource 844424930131969
      push Request 844424930131969
      push Job 844424930131968
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      ... 4 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      ... 6 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      ... 7 more
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more
      Sep 30, 2015 12:40:35 PM org.apache.hyracks.control.nc.Joblet close
      WARNING: Freeing leaked 1507328 bytes
      Sep 30, 2015 12:40:35 PM org.apache.hyracks.control.nc.Joblet close
      WARNING: Freeing leaked 1638400 bytes
      org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
      at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:45)
      at org.apache.hyracks.control.nc.Task.run(Task.java:312)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      ... 4 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      ... 6 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      ... 7 more
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more
      Sep 30, 2015 12:40:35 PM org.apache.asterix.api.http.servlet.RESTAPIServlet handleRequest
      SEVERE: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
      at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:45)
      at org.apache.hyracks.control.nc.Task.run(Task.java:312)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      ... 4 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      ... 6 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      ... 7 more
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more

      Sep 30, 2015 12:40:35 PM org.apache.asterix.test.aql.TestsUtils executeHttpMethod
      SEVERE: org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
      at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:45)
      at org.apache.hyracks.control.nc.Task.run(Task.java:312)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      ... 4 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      ... 6 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      ... 7 more
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more

      testFile src/test/resources/runtimets/queries/dml/insert-delete-using-secondary-index/insert-delete-using-secondary-index.3.update.aql raised an exception:
      java.lang.Exception: HTTP operation failed: 99
      STATUS LINE: HTTP/1.1 500 Server Error
      SUMMARY: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      caused by: org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
      caused by: org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:45)
      caused by: org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      caused by: org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      caused by: org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      caused by: org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      caused by: org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      STACKTRACE: org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
      org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
      at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:45)
      at org.apache.hyracks.control.nc.Task.run(Task.java:312)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:358)
      at org.apache.hyracks.control.nc.Task.run(Task.java:290)
      ... 3 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:80)
      at org.apache.hyracks.dataflow.std.misc.MaterializingOperatorDescriptor$MaterializerReaderActivityNode$1.close(MaterializingOperatorDescriptor.java:117)
      at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
      ... 4 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:122)
      at org.apache.hyracks.dataflow.std.misc.MaterializerTaskState.writeOut(MaterializerTaskState.java:75)
      ... 6 more
      Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:62)
      at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:336)
      at org.apache.hyracks.storage.am.btree.impls.BTree.access$5(BTree.java:329)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:901)
      at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsert(BTree.java:894)
      at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:370)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:335)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:325)
      at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.delete(LSMTreeIndexAccessor.java:62)
      at org.apache.asterix.common.dataflow.AsterixLSMInsertDeleteOperatorNodePushable.nextFrame(AsterixLSMInsertDeleteOperatorNodePushable.java:108)
      ... 7 more
      Caused by: org.apache.asterix.common.exceptions.ACIDException: Transaction JID:277 should abort (requested by the Lock Manager):
      Job 3:0:0
      Resource 3:0:1
      Request 3:0:1
      Job 3:0:0

      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.requestAbort(ConcurrentLockManager.java:922)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.enqueueWaiter(ConcurrentLockManager.java:179)
      at org.apache.asterix.transaction.management.service.locking.ConcurrentLockManager.lock(ConcurrentLockManager.java:154)
      at org.apache.asterix.transaction.management.opcallbacks.PrimaryIndexModificationOperationCallback.before(PrimaryIndexModificationOperationCallback.java:53)
      ... 16 more

      at org.apache.asterix.test.aql.TestsUtils.executeHttpMethod(TestsUtils.java:213)
      at org.apache.asterix.test.aql.TestsUtils.executeUpdate(TestsUtils.java:247)
      at org.apache.asterix.test.aql.TestsUtils.executeTest(TestsUtils.java:427)
      at org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:144)
      at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      at org.junit.runners.Suite.runChild(Suite.java:127)
      at org.junit.runners.Suite.runChild(Suite.java:26)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
      at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
      at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

      Attachments

        Activity

          People

            tillw Till Westmann
            wangsaeu Taewoo Kim
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: