Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-19278

AND queries involving both static and regular columns can hide matches from Memtable-attached indexes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 5.0-beta2, 5.0, 5.x
    • Feature/SAI
    • None

    Description

      As part of our ongoing work on CASSANDRA-18275, Harry started surfacing errors shortly after we added static columns to the schema of SingleNodeSAITest. After a bit of debugging, I was able to come up with the following concrete repro:

      import org.junit.Test;
      
      import org.apache.cassandra.cql3.UntypedResultSet;
      import org.apache.cassandra.index.sai.utils.SAIRandomizedTester;
      
      public class InMemoryIntersectionsTest extends SAIRandomizedTester
      {
          @Test
          public void staticAndRegularIntersection()
          {
              createTable("CREATE TABLE %s (pk int, ck int, v1 int, s1 int static, PRIMARY KEY(pk, ck))");
              createIndex("CREATE INDEX ON %s(v1) USING 'sai'");
              createIndex("CREATE INDEX ON %s(s1) USING 'sai'");
      
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 1, 0);
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 2, 1);
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 3, 2);
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 4, 3);
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 5, 4);
              execute("INSERT INTO %s (pk, ck, v1) VALUES (?, ?, ?)", 0, 6, 5);
      
              execute("INSERT INTO %s (pk, s1) VALUES (?, ?)", 0, 100);
      
      	// Flushing here passes test
      
              UntypedResultSet result1 = execute("SELECT * FROM %s WHERE pk = ? AND v1 > ?", 0, 2);
              assertRowCount(result1, 3);
              UntypedResultSet result2 = execute("SELECT * FROM %s WHERE pk = ? AND v1 > ? AND s1 = ?", 0, 2, 100);        
              assertRowCount(result2, 3); // Only returns one result!
          }
      }
      

      Flushing memtables immediately before the queries passes the test, but it seems we have an issue with how the PrimaryKey iterators produced by static and regular column indexes are intersected when the iterators come from Memtable-attached indexes. Once the root cause of this is determined, it might make some sense to enhance RandomIntersectionTest to cover this interaction as well, just in case it turns up further problems.

      Attachments

        1. ci_summary.html
          7 kB
          Caleb Rackliffe
        2. ci_summary-1.html
          7 kB
          Caleb Rackliffe
        3. result_details.tar.gz
          37.61 MB
          Caleb Rackliffe
        4. result_details.tar-1.gz
          39.84 MB
          Caleb Rackliffe

        Activity

          People

            maedhroz Caleb Rackliffe
            maedhroz Caleb Rackliffe
            Caleb Rackliffe, Piotr Kolaczkowski
            Alex Petrov, Piotr Kolaczkowski
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 40m
                4h 40m