Cassandra
  1. Cassandra
  2. CASSANDRA-4297

Use java NIO as much as possible when streaming compressed SSTables

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 1
    • Component/s: None
    • Labels:

      Description

      Back in 0.8, streaming uses java NIO (FileChannel#transferTo/transferFrom) to perform zero copy file transfer between nodes. Since 1.0, in order to add new features like sstable compression and internode encryption we had to switch to java IO Input/OutputStreams. What we currently do to transfer compressed SSTable is, in source node, 1) decompress chunk in SSTable, 2) compress using LZF for network, and in destination node, 3) decompress using LZF as reading from socket, 4) compress for SSTable on disk.

      Now, 1.1 comes out with SSTable compression turned on by default. It is reasonable to transfer compressed file as is using NIO instead of decompress/compress in source node.

      1. 0001-Use-SortedSet-instead-of-Set-and-Arrays.sort.txt
        2 kB
        Yuki Morishita
      2. 4297.txt
        38 kB
        Yuki Morishita
      3. 4297-v2.txt
        44 kB
        Yuki Morishita

        Activity

          People

          • Assignee:
            Yuki Morishita
            Reporter:
            Yuki Morishita
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development