Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16772

User Defined nodetool cleanup only processes one SSTable per table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • 3.11.x
    • Local/SSTable
    • None
    • Correctness
    • Normal
    • Normal
    • User Report
    • All
    • None

    Description

      User defined nodetool cleanup uses a HashMap instead of a MultiMap to group the user provided SSTables by table.  This means it only keeps one file per source table.

      It also means the unit test for this component is not sufficient.

      As part of https://issues.apache.org/jira/browse/CASSANDRA-16767  I introduced a helper method on Descriptor:

      public static Multimap<ColumnFamilyStore, Descriptor> fromFilenamesGrouped(Collection<String> filenames) 

      That should be used instead of the custom logic in CompactionManager.forceUserDefinedCleanup.

       

      Broken existing code:

              HashMap<ColumnFamilyStore, Descriptor> descriptors = Maps.newHashMap();        for (String filename : filenames)
              {
                  // extract keyspace and columnfamily name from filename
                  Descriptor desc = Descriptor.fromFilename(filename.trim());
                  if (Schema.instance.getCFMetaData(desc) == null)
                  {
                      logger.warn("Schema does not exist for file {}. Skipping.", filename);
                      continue;
                  }
                  // group by keyspace/columnfamily
                  ColumnFamilyStore cfs = Keyspace.open(desc.ksname).getColumnFamilyStore(desc.cfname);
                  desc = cfs.getDirectories().find(new File(filename.trim()).getName());
                  if (desc != null)
                      descriptors.put(cfs, desc);
              } 

       

      Contents of helper method introduced in other ticket:

       public static Multimap<ColumnFamilyStore, Descriptor> fromFilenamesGrouped(Collection<String> filenames) {
            Multimap<ColumnFamilyStore, Descriptor> descriptors = ArrayListMultimap.create();      for (String filename : filenames)
            {
                // extract keyspace and columnfamily name from filename
                Descriptor desc = Descriptor.fromFilename(filename.trim());
                if (Schema.instance.getCFMetaData(desc) == null)
                {
                    logger.warn("Schema does not exist for file {}. Skipping.", filename);
                    continue;
                }
                // group by keyspace/columnfamily
                ColumnFamilyStore cfs = Keyspace.open(desc.ksname).getColumnFamilyStore(desc.cfname);
                desc = cfs.getDirectories().find(new File(filename.trim()).getName());
                if (desc != null)
                  descriptors.put(cfs, desc);
            }
            return descriptors;
          } 

      Attachments

        Activity

          People

            scottcarey Scott Carey
            scottcarey Scott Carey
            Scott Carey
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: