Cassandra
  1. Cassandra
  2. CASSANDRA-4297

Use java NIO as much as possible when streaming compressed SSTables

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 1.2.0 beta 1
    • Component/s: None
    • Labels:

      Description

      Back in 0.8, streaming uses java NIO (FileChannel#transferTo/transferFrom) to perform zero copy file transfer between nodes. Since 1.0, in order to add new features like sstable compression and internode encryption we had to switch to java IO Input/OutputStreams. What we currently do to transfer compressed SSTable is, in source node, 1) decompress chunk in SSTable, 2) compress using LZF for network, and in destination node, 3) decompress using LZF as reading from socket, 4) compress for SSTable on disk.

      Now, 1.1 comes out with SSTable compression turned on by default. It is reasonable to transfer compressed file as is using NIO instead of decompress/compress in source node.

      1. 4297.txt
        38 kB
        Yuki Morishita
      2. 4297-v2.txt
        44 kB
        Yuki Morishita
      3. 0001-Use-SortedSet-instead-of-Set-and-Arrays.sort.txt
        2 kB
        Yuki Morishita

        Activity

        Yuki Morishita created issue -
        Jonathan Ellis made changes -
        Field Original Value New Value
        Reviewer jbellis
        Yuki Morishita made changes -
        Attachment 4297.txt [ 12531114 ]
        Yuki Morishita made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Yuki Morishita made changes -
        Attachment 4297-v2.txt [ 12531866 ]
        Yuki Morishita made changes -
        Yuki Morishita made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12670764 ] patch-available, re-open possible [ 12749809 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12749809 ] reopen-resolved, no closed status, patch-avail, testing [ 12757323 ]

          People

          • Assignee:
            Yuki Morishita
            Reporter:
            Yuki Morishita
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development