Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-1872

java.lang.IndexOutOfBoundsException when ingesting data using small memory component size

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      I was testing LSM merge policies recently, and thus I used small memory component size (storage.memorycomponent.numpages=8, storage.buffercache.pagesize=128kb). However, when ingesting some sample tweets, I kept receiving java.lang.IndexOutOfBoundsException

      java.lang.IndexOutOfBoundsException
      	at java.nio.Buffer.checkIndex(Buffer.java:546)
      	at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:365)
      	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.isInner(LIFOMetaDataFrame.java:210)
      	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.get(LIFOMetaDataFrame.java:174)
      	at org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.get(AppendOnlyLinkedMetadataPageManager.java:332)
      	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.getTreeIndexLSN(AbstractLSMIOOperationCallback.java:106)
      	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.getComponentLSN(LSMBTreeIOOperationCallback.java:72)
      	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.putLSNIntoMetadata(AbstractLSMIOOperationCallback.java:100)
      	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.afterOperation(LSMBTreeIOOperationCallback.java:47)
      	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:519)
      	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:112)
      	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:83)
      	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:1)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Is this behavior expected?

      Steps to reproduce
      1. I started a simple cluster with one CC and one NC using org.apache.asterix.api.common.AsterixHyracksIntegrationUtil, and the configuration file is attached.

      2. The sample tweets I used is from "https://github.com/ISG-ICS/cloudberry".
      The tweets can be ingested by entering the root directory of the project and typing "./script/ingestTwitterToLocalCluster.sh".
      Note: please use the modified ingestTwitterToLocalCluster.sh attached.

        Attachments

        1. ingestTwitterToLocalCluster.sh
          3 kB
          Chen Luo
        2. asterix-build-configuration-lsm.xml
          3 kB
          Chen Luo

          Issue Links

            Activity

              People

              • Assignee:
                imaxon Ian Maxon
                Reporter:
                luochen01 Chen Luo
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: