Cassandra
  1. Cassandra
  2. CASSANDRA-4282

Improve startup time by making row cache population multi threaded

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 1
    • Component/s: None
    • Labels:
      None

      Description

      Attached patch multi threads the row cache population

      my small tests show improvements atleast;

      - 4 cores                                                                                                                                                                                              
      - 11G stress-generated data
      - page cache dropped between runs                                                                                                                                                                            
                                                                                                                                                                                                             
      single platter spinning disk:                                                                                                                                                   
      single threaded:                                                                                                                                                                                       
       INFO 10:18:21,365 completed loading (245562 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
       INFO 10:32:43,738 completed loading (255106 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
      multi threaded:                                                                                                                                                                                        
       INFO 10:22:47,567 completed loading (213905 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
       INFO 10:27:26,873 completed loading (214514 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
                                                                                                                                                                                                             
      ssd;                                                                                                                                                                                                   
      single threaded:                                                                                                                                                                                       
       INFO 10:40:49,079 completed loading (103883 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
       INFO 10:43:45,799 completed loading (106913 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                        
      multi threaded:                                                                                                                                                                                        
       INFO 10:38:20,798 completed loading (57617 ms; 311381 keys) row cache for Keyspace1.Standard1                                                                                                         
       INFO 10:47:20,339 completed loading (56682 ms; 311381 keys) row cache for Keyspace1.Standard1
      

        Activity

        Hide
        Marcus Eriksson added a comment -

        btw, unsure how this relates to CASSANDRA-3762, might be conflicts

        Show
        Marcus Eriksson added a comment - btw, unsure how this relates to CASSANDRA-3762 , might be conflicts
        Hide
        Jonathan Ellis added a comment -

        Sorry for the delay reviewing. I'm unable to apply to current trunk or trunk as of May 25... could you rebase?

        Show
        Jonathan Ellis added a comment - Sorry for the delay reviewing. I'm unable to apply to current trunk or trunk as of May 25... could you rebase?
        Hide
        Jonathan Ellis added a comment -

        v2 takes the approach of modifying CacheSerializer.deserialize to return a Future. The row values are read on the read Stage instead of creating a new ExecutorService.

        Show
        Jonathan Ellis added a comment - v2 takes the approach of modifying CacheSerializer.deserialize to return a Future. The row values are read on the read Stage instead of creating a new ExecutorService.
        Hide
        Vijay added a comment -

        +1

        Show
        Vijay added a comment - +1
        Hide
        Jonathan Ellis added a comment -

        committed

        Show
        Jonathan Ellis added a comment - committed

          People

          • Assignee:
            Jonathan Ellis
            Reporter:
            Marcus Eriksson
            Reviewer:
            Vijay
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development