Cassandra
  1. Cassandra
  2. CASSANDRA-1470

Avoid polluting page cache during compaction

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 0.7.1
    • Component/s: Core
    • Labels:
      None

      Description

      When compaction scans through a group of sstables, it forces the data in the os buffer cache being used for hot reads, which can have a dramatic negative effect on performance.

      1. 1470.txt
        30 kB
        Jonathan Ellis
      2. 1470-v2.txt
        23 kB
        Jonathan Ellis
      3. CASSANDRA-1470.patch
        7 kB
        Pavel Yaskevich
      4. CASSANDRA-1470-for-0.6.patch
        7 kB
        Pavel Yaskevich
      5. CASSANDRA-1470-v10-for-0.7.patch
        37 kB
        T Jake Luciani
      6. CASSANDRA-1470-v11-for-0.7.patch
        38 kB
        T Jake Luciani
      7. CASSANDRA-1470-v12-0.7.patch
        36 kB
        Pavel Yaskevich
      8. CASSANDRA-1470-v13-0.7.patch
        36 kB
        Pavel Yaskevich
      9. CASSANDRA-1470-v14-0.7.patch
        34 kB
        T Jake Luciani
      10. CASSANDRA-1470-v2.patch
        8 kB
        Pavel Yaskevich
      11. CASSANDRA-1470-v3-0.7-with-LastErrorException-support.patch
        8 kB
        Pavel Yaskevich
      12. CASSANDRA-1470-v4-for-0.7.patch
        17 kB
        Pavel Yaskevich
      13. CASSANDRA-1470-v5-for-0.7.patch
        30 kB
        Pavel Yaskevich
      14. CASSANDRA-1470-v6-for-0.7.patch
        31 kB
        Pavel Yaskevich
      15. CASSANDRA-1470-v7-for-0.7.patch
        32 kB
        T Jake Luciani
      16. CASSANDRA-1470-v8-for-0.7.patch
        33 kB
        Pavel Yaskevich
      17. CASSANDRA-1470-v9-for-0.7.patch
        33 kB
        Pavel Yaskevich
      18. use.DirectIORandomAccessFile.for.commitlog.against.1022235.patch
        1.0 kB
        Robert Coli

        Issue Links

          Activity

          Jonathan Ellis created issue -
          Jonathan Ellis made changes -
          Field Original Value New Value
          Attachment 1470.txt [ 12453883 ]
          Jonathan Ellis made changes -
          Attachment 1470-v2.txt [ 12453884 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.6.7 [ 12315375 ]
          Fix Version/s 0.7.0 [ 12315212 ]
          Fix Version/s 0.6.6 [ 12315264 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.6.7 [ 12315375 ]
          Fix Version/s 0.7.1 [ 12315199 ]
          Fix Version/s 0.7.0 [ 12315212 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.6.7 [ 12315375 ]
          Robert Coli made changes -
          Jonathan Ellis made changes -
          Assignee Pavel Yaskevich [ xedin ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470.patch [ 12458543 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Reviewer jbellis
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v2.patch [ 12458554 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Pavel Yaskevich made changes -
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-for-0.6.patch [ 12458624 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v4-for-0.7.patch [ 12458931 ]
          Pavel Yaskevich made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v5-for-0.7.patch [ 12459072 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v5-for-0.7.patch [ 12459072 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v5-for-0.7.patch [ 12459098 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.6.8 [ 12315525 ]
          Fix Version/s 0.7.1 [ 12315199 ]
          Fix Version/s 0.6.7 [ 12315375 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.6.9 [ 12315556 ]
          Fix Version/s 0.6.8 [ 12315525 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v6-for-0.7.patch [ 12459470 ]
          Pavel Yaskevich made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          T Jake Luciani made changes -
          Attachment CASSANDRA-1470-v7-for-0.7.patch [ 12459673 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v8-for-0.7.patch [ 12459690 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v9-for-0.7.patch [ 12459699 ]
          T Jake Luciani made changes -
          Attachment CASSANDRA-1470-v10-for-0.7.patch [ 12459761 ]
          T Jake Luciani made changes -
          Attachment CASSANDRA-1470-v11-for-0.7.patch [ 12459836 ]
          Jonathan Ellis made changes -
          Fix Version/s 0.7.1 [ 12315199 ]
          Fix Version/s 0.6.9 [ 12315556 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] In Progress [ 3 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v12-for-0.7.patch [ 12465695 ]
          Pavel Yaskevich made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v12-for-0.7.patch [ 12465695 ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v12-0.7.patch [ 12465790 ]
          Pavel Yaskevich made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Pavel Yaskevich made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Oleg Anastasyev made changes -
          Comment [ Guys, may be I misunderstand something, but are we trying to make compaction writes faster ? This is background data maintenance process, and I think it is not very much important how fast is it. I think most important about it is how much it makes normal (read) requests serving slower, keeping valuable resources, especially memory, busy.
          Did we measured the impact of compaction on read requests of direct IO writes compaction vs old normal writes compaction ?

          ]
          Pavel Yaskevich made changes -
          Attachment CASSANDRA-1470-v13-0.7.patch [ 12466151 ]
          Pavel Yaskevich made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          T Jake Luciani made changes -
          Attachment CASSANDRA-1470-v14-0.7.patch [ 12466252 ]
          Jonathan Ellis made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Reviewer jbellis tjake
          Resolution Fixed [ 1 ]
          T Jake Luciani made changes -
          Summary use direct io for compaction Avoid polluting page cache during compaction
          Hiram Chirino made changes -
          Link This issue is related to CASSANDRA-1908 [ CASSANDRA-1908 ]
          Jonathan Ellis made changes -
          Original Estimate 80h [ 288000 ]
          Remaining Estimate 80h [ 288000 ]
          Gavin made changes -
          Workflow no-reopen-closed, patch-avail [ 12519731 ] patch-available, re-open possible [ 12753512 ]
          Gavin made changes -
          Workflow patch-available, re-open possible [ 12753512 ] reopen-resolved, no closed status, patch-avail, testing [ 12756684 ]

            People

            • Assignee:
              Pavel Yaskevich
              Reporter:
              Jonathan Ellis
              Reviewer:
              T Jake Luciani
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 80h
                80h
                Remaining:
                Remaining Estimate - 80h
                80h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development