Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 2
    • Component/s: Core
    • Labels:
      None

      Description

      Bloom filters are the major user of heap as dataset grows. It's probably worth it to move these off heap. No extra refcounting needs to be done since we already refcount SSTableReader.

      1. 0001-CASSANDRA-4865-v3.patch
        49 kB
        Vijay
      2. 0001-CASSANDRA-4865-v2.patch
        47 kB
        Vijay
      3. 0001-CASSANDRA-4865.patch
        46 kB
        Vijay

        Activity

        Hide
        Jonathan Ellis added a comment -

        Vijay, could you take a stab at this? The bit arithmetic made my head hurt. (may want to add a getInt method to Memory to avoid signed/unsigned messes... then again maybe getByte is faster from native, no idea really.)

        Show
        Jonathan Ellis added a comment - Vijay, could you take a stab at this? The bit arithmetic made my head hurt. (may want to add a getInt method to Memory to avoid signed/unsigned messes... then again maybe getByte is faster from native, no idea really.)
        Hide
        Vijay added a comment -

        Will do thanks!

        Show
        Vijay added a comment - Will do thanks!
        Hide
        Vijay added a comment -

        Good news, The new implementation uses byte's instead of longs which gives us approximately 20% better performance. In Practice it should save some GC time too.

        Time taken for approx 200 Million iterations each.

        Open BS set's Open BS get's Offheap BS set's Offheap BS get's
        502 371 311 64
        507 444 257 366
        496 478 310 367
        504 473 306 369
        490 481 305 367
        502 472 314 363
        489 476 305 367
        486 474 303 364
        489 474 307 365
        492 477 305 365
        490 475 307 367

        Attached patch enables offheap BS only for SSTable BF's and leaves the Promoted indexes alone.
        Attached patch will break the existing 1.2-beta users, New BF serialization is changed. (is that expected or ok to do?)

        Pending/ TODO's:
        1) have to figure out why Scrub test trys to free memory twice.
        2) Have to regenerate Corrupted SST's (unit test failure).

        Show
        Vijay added a comment - Good news, The new implementation uses byte's instead of longs which gives us approximately 20% better performance. In Practice it should save some GC time too. Time taken for approx 200 Million iterations each. Open BS set's Open BS get's Offheap BS set's Offheap BS get's 502 371 311 64 507 444 257 366 496 478 310 367 504 473 306 369 490 481 305 367 502 472 314 363 489 476 305 367 486 474 303 364 489 474 307 365 492 477 305 365 490 475 307 367 Attached patch enables offheap BS only for SSTable BF's and leaves the Promoted indexes alone. Attached patch will break the existing 1.2-beta users, New BF serialization is changed. (is that expected or ok to do?) Pending/ TODO's: 1) have to figure out why Scrub test trys to free memory twice. 2) Have to regenerate Corrupted SST's (unit test failure).
        Hide
        Jonathan Ellis added a comment -

        Nit: could use unsafe.setMemory as a faster clear

        Show
        Jonathan Ellis added a comment - Nit: could use unsafe.setMemory as a faster clear
        Hide
        Vijay added a comment - - edited

        Attached patch includes the nit and updated tests.

        re-ran the test on 32 cores and shows bigger diffrence.

        Open BS set's Open BS get's Open BS clear's Offheap BS set's Offheap BS get's Offheap BS clear's
        593 495 658 408 132 392
        606 479 749 394 130 556
        542 452 736 394 130 554
        543 460 736 395 129 555
        542 452 736 391 129 554
        541 465 735 391 129 555
        542 454 737 392 129 555
        541 452 736 391 130 554
        542 452 735 391 129 554
        542 452 736 391 130 554
        542 519 736 391 130 554

        ran stress just for fun
        before: aprox 2750/sec
        after: aprox 2800/sec

        Its a big win for a small change i guess

        Show
        Vijay added a comment - - edited Attached patch includes the nit and updated tests. re-ran the test on 32 cores and shows bigger diffrence. Open BS set's Open BS get's Open BS clear's Offheap BS set's Offheap BS get's Offheap BS clear's 593 495 658 408 132 392 606 479 749 394 130 556 542 452 736 394 130 554 543 460 736 395 129 555 542 452 736 391 129 554 541 465 735 391 129 555 542 454 737 392 129 555 541 452 736 391 130 554 542 452 735 391 129 554 542 452 736 391 130 554 542 519 736 391 130 554 ran stress just for fun before: aprox 2750/sec after: aprox 2800/sec Its a big win for a small change i guess
        Hide
        Jonathan Ellis added a comment -

        Would prefer to not need to rewrite existing sstables to get off-heap BF – seems to me we can make the off heap de/serialize code compatible with the original OpenBitSet code. Slightly ugly to have the unnecessary pageCount/Size business but worth it to get instant off-heap BF on upgrade.

        Show
        Jonathan Ellis added a comment - Would prefer to not need to rewrite existing sstables to get off-heap BF – seems to me we can make the off heap de/serialize code compatible with the original OpenBitSet code. Slightly ugly to have the unnecessary pageCount/Size business but worth it to get instant off-heap BF on upgrade.
        Hide
        Vijay added a comment -

        Done, Thanks!

        Show
        Vijay added a comment - Done, Thanks!
        Hide
        Jonathan Ellis added a comment -

        Committed! Nice work getting this done for beta2.

        Show
        Jonathan Ellis added a comment - Committed! Nice work getting this done for beta2.

          People

          • Assignee:
            Vijay
            Reporter:
            Jonathan Ellis
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development