Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      CASSANDRA-2995 proposes full storage engine pluggability, which is probably unavoidable in the long run. For now though, I'd like to propose an incremental alternative that preserves the sstable model, but allows it to evolve non-linearly.

      The sstable "version" field could allow for simple switching between writable sstable types, without moving all the way to differentiating between engines as CASSANDRA-2995 requires. This can be accomplished by moving towards a "feature flags" model (with a mapping between versions and feature sets), rather than a linear versions model (where versions can be strictly ordered and all versions above X have a feature).

      There are restrictions on this approach:

      • It's sufficient for an alternate SSTable(Writer|Reader|*) set to require a patch to enable (rather than a JAR)
      • Filenames/descriptors/components must conform to the existing conventions

        Issue Links

          Activity

          Hide
          stuhood Stu Hood added a comment -

          Patchset to create all necessary abstract base classes for SSTable pluggability.

          0002 and 0006 deal with allowing for non-linear SSTable versions, and overriding the default.

          This set applies atop CASSANDRA-2629 but should be ready for high level review.

          Show
          stuhood Stu Hood added a comment - Patchset to create all necessary abstract base classes for SSTable pluggability. 0002 and 0006 deal with allowing for non-linear SSTable versions, and overriding the default. This set applies atop CASSANDRA-2629 but should be ready for high level review.
          Hide
          slebresne Sylvain Lebresne added a comment -

          Had an initial look at the patches. I haven't try applying the patches yet (which likely need rebase anyway), so following comments are fairly high level.

          For the abstract interfaces to SSTable

          {Reader, Writer, *Iterator}

          , I would prefer calling them AbstractSSTable* and keep the concrete implementation with their current name. It should make the diffs much more digestible and much more easy to check. Same thing for future merges. And we kind of use Abstract* for abstract place around the place so it would rather more consistent than less.

          For patch 2, I'm not too sure about what this refactoring buys us, in particular as long as files are versioned by one character. I understand that to support other kind of sstables, we would need to somehow distinguish between "kind" of sstables, but this doesn't seem to fully solve that problem. I think we already had kind of the same problem with compression, where we went with using the presence of a sstable component to decide if it's compressed or not. I'm happy to have a discussion on how to best evolve the sstable versioning but not sure about the solution here. Follows the same holds for patch 6.

          Small nitpick: In patch1, I'm not too fond of introducing a compareTo for SSTableIdentityIterator (given a SSTableIdentityIterator is not really equal to it's key), is there an underlying reason ?

          Show
          slebresne Sylvain Lebresne added a comment - Had an initial look at the patches. I haven't try applying the patches yet (which likely need rebase anyway), so following comments are fairly high level. For the abstract interfaces to SSTable {Reader, Writer, *Iterator} , I would prefer calling them AbstractSSTable* and keep the concrete implementation with their current name. It should make the diffs much more digestible and much more easy to check. Same thing for future merges. And we kind of use Abstract* for abstract place around the place so it would rather more consistent than less. For patch 2, I'm not too sure about what this refactoring buys us, in particular as long as files are versioned by one character. I understand that to support other kind of sstables, we would need to somehow distinguish between "kind" of sstables, but this doesn't seem to fully solve that problem. I think we already had kind of the same problem with compression, where we went with using the presence of a sstable component to decide if it's compressed or not. I'm happy to have a discussion on how to best evolve the sstable versioning but not sure about the solution here. Follows the same holds for patch 6. Small nitpick: In patch1, I'm not too fond of introducing a compareTo for SSTableIdentityIterator (given a SSTableIdentityIterator is not really equal to it's key), is there an underlying reason ?
          Hide
          slebresne Sylvain Lebresne added a comment -

          Closing this for lack of progress. I'm also not sure this is buying us much in practice in itself. Feel free to re-open if you want to revisit this.

          Show
          slebresne Sylvain Lebresne added a comment - Closing this for lack of progress. I'm also not sure this is buying us much in practice in itself. Feel free to re-open if you want to revisit this.

            People

            • Assignee:
              stuhood Stu Hood
              Reporter:
              stuhood Stu Hood
              Reviewer:
              Sylvain Lebresne
            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development