Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.0.9
    • Component/s: Core
    • Labels:
      None
    • Environment:

      We start getting this exception after upgrading from 1.0.1 -> 1.0.8.

      4 nodes cluster on Cassandra v1.0.8. RF = 3.
      Hector v0.8.0-3.

      Description

      We have a column family with String row keys and Long column keys.

      Our WideEntityService is trying to get the first column in the range from 0 to Long.MAX. It's a batch operation performed for every row in the CF (rows count is approximately tens of thousands and each row contains from 0 to 1000 columns).

      After processing each row we are removing some of the columns we have queried. Also, at the same time we are writing in this CF in another threads but somewhat less intensive.

      An error rises approximately for a one of 100 rows.

      Exception itself:
      [05-Mar-2012 18:47:25,247] ERROR [http-8095-1 WideEntityServiceImpl.java:142] - get: key1 -

      {type=RANGE, start=0, end=9223372036854775807, orderDesc=false, limit=1}

      me.prettyprint.hector.api.exceptions.HCassandraInternalException: Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error processing get_slice
      at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:31)
      at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
      at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
      at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
      at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:233)
      at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
      at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289)
      at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53)
      at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49)
      at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
      at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
      at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48)

      1. 4000.txt
        1 kB
        Sylvain Lebresne

        Activity

        Sergey B created issue -
        Sergey B made changes -
        Field Original Value New Value
        Environment We start getting this exception after upgrading from 1.0.1 -> 1.0.8.

        4 nodes cluster on Cassandra v1.0.8. RF = 3.
        Hector v0.8.0-3.

        We have a column family with String row keys and Long column keys.

        Our WideEntityService is trying to get the first column in the range from 0 to Long.MAX. It's a batch operation performed for every row in the CF (rows count is approximately tens of thousands and each row contains from 0 to 1000 columns).

        After processing each row we are removing some of the columns we have queried. Also, at the same time we are writing in this CF in another threads but somewhat less intensive.

        An error rises approximately for a one of 100 rows.
        We start getting this exception after upgrading from 1.0.1 -> 1.0.8.

        4 nodes cluster on Cassandra v1.0.8. RF = 3.
        Hector v0.8.0-3.
        Description [05-Mar-2012 18:47:25,247] ERROR [http-8095-1 WideEntityServiceImpl.java:142] - get: key1 - {type=RANGE, start=0, end=9223372036854775807, orderDesc=false, limit=1}
        me.prettyprint.hector.api.exceptions.HCassandraInternalException: Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error processing get_slice
                at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:31)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
                at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
                at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:233)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49)
                at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
                at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48)
        We have a column family with String row keys and Long column keys.

        Our WideEntityService is trying to get the first column in the range from 0 to Long.MAX. It's a batch operation performed for every row in the CF (rows count is approximately tens of thousands and each row contains from 0 to 1000 columns).

        After processing each row we are removing some of the columns we have queried. Also, at the same time we are writing in this CF in another threads but somewhat less intensive.

        An error rises approximately for a one of 100 rows.


        Exception itself:
        [05-Mar-2012 18:47:25,247] ERROR [http-8095-1 WideEntityServiceImpl.java:142] - get: key1 - {type=RANGE, start=0, end=9223372036854775807, orderDesc=false, limit=1}
        me.prettyprint.hector.api.exceptions.HCassandraInternalException: Cassandra encountered an internal error processing this request: TApplicationError type: 6 message:Internal error processing get_slice
                at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:31)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:285)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl$7.execute(KeyspaceServiceImpl.java:268)
                at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:101)
                at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:233)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
                at me.prettyprint.cassandra.service.KeyspaceServiceImpl.getSlice(KeyspaceServiceImpl.java:289)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:53)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery$1.doInKeyspace(ThriftSliceQuery.java:49)
                at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
                at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
                at me.prettyprint.cassandra.model.thrift.ThriftSliceQuery.execute(ThriftSliceQuery.java:48)
        Sylvain Lebresne made changes -
        Assignee Sylvain Lebresne [ slebresne ]
        Sylvain Lebresne made changes -
        Attachment 4000.txt [ 12517216 ]
        Sylvain Lebresne made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 1.0.9 [ 12319856 ]
        Sylvain Lebresne made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Reviewer jbellis
        Resolution Fixed [ 1 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12656640 ] patch-available, re-open possible [ 12749606 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12749606 ] reopen-resolved, no closed status, patch-avail, testing [ 12754322 ]

          People

          • Assignee:
            Sylvain Lebresne
            Reporter:
            Sergey B
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development