Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-1214

Force linux to not swap the JVM

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Fix Version/s: 0.6.5, 0.7 beta 2
    • Component/s: None
    • Labels:
      None

      Description

      The way mmap()'d IO is handled in cassandra is dangerous. It allocates potentially massive buffers without any care for bounding the total size of the program's buffers. As the node's dataset grows, this will lead to swapping and instability.

      This is a dangerous and wrong default for a couple of reasons.

      1) People are likely to test cassandra with the default settings. This issue is insidious because it only appears when you have sufficient data in a certain node, there is absolutely no way to control it, and it doesn't at all respect the memory limits that you give to the JVM.

      That can all be ascertained by reading the code, and people should certainly do their homework, but nevertheless, cassandra should ship with sane defaults that don't break down when you cross some magic unknown threshold.

      2) It's deceptive. Unless you are extremely careful with capacity planning, you will get bit by this. Most people won't really be able to use this in production, so why get them excited about performance that they can't actually have?

        Attachments

        1. trunk-1214.txt
          13 kB
          Peter Schuller
        2. Read Throughput with mmap.jpg
          36 kB
          Schubert Zhang
        3. mlockall-jna.patch.txt
          2 kB
          Folke Behrens
        4. 1214-v4.txt
          10 kB
          Jonathan Ellis
        5. 1214-v3.txt
          8 kB
          Jonathan Ellis

          Activity

            People

            • Assignee:
              jbellis Jonathan Ellis
              Reporter:
              jamesgolick James Golick
              Authors:
              Jonathan Ellis
              Reviewers:
              Jon Hermes
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: