Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-4297

Use java NIO as much as possible when streaming compressed SSTables

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 1
    • Component/s: None
    • Labels:

      Description

      Back in 0.8, streaming uses java NIO (FileChannel#transferTo/transferFrom) to perform zero copy file transfer between nodes. Since 1.0, in order to add new features like sstable compression and internode encryption we had to switch to java IO Input/OutputStreams. What we currently do to transfer compressed SSTable is, in source node, 1) decompress chunk in SSTable, 2) compress using LZF for network, and in destination node, 3) decompress using LZF as reading from socket, 4) compress for SSTable on disk.

      Now, 1.1 comes out with SSTable compression turned on by default. It is reasonable to transfer compressed file as is using NIO instead of decompress/compress in source node.

        Attachments

        1. 0001-Use-SortedSet-instead-of-Set-and-Arrays.sort.txt
          2 kB
          Yuki Morishita
        2. 4297-v2.txt
          44 kB
          Yuki Morishita
        3. 4297.txt
          38 kB
          Yuki Morishita

          Activity

            People

            • Assignee:
              yukim Yuki Morishita
              Reporter:
              yukim Yuki Morishita
              Reviewer:
              Jonathan Ellis
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: