Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15179

multiple validation compactions on single table

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Triage Needed
    • Priority: Normal
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Consistency/Repair
    • Labels:
      None
    • Platform:
      All
    • Impacts:
      None

      Description

      In a 2 node cluster I'm observing that after a repair is started, any repairs started leaves validation compactions running on the coordinator node.

      The non-coordinator correctly rejects the repair with

      ActiveRepairService.java:525 - Cannot start multiple repair sessions over the same sstables

      but the coordinator did not. When testing to confirm the behavior I was able to increase the executing validations from 11 to ~18 running on the same table and an additional 17 [compactions] pending with high confidence they were pending validations. (see attached scrubbed files).

      Repairs were started with: nodetool repair – keyspace_01 table_02

      Initially I thought this might be fixed with CASSANDRA-14332, but that is a revert of changes from CASSANDRA-13797 which was released in 3.0.15 and I'm running 3.0.10 so shouldn't be affected. So far in the ~1.5 hours since noticing this I haven't been able to recreate the issue.

      I'm will soon be validating 3.0.18 and will retest with that release if I can figure out how to recreate.

      Environment Info:

      • Cassandra 3.0.10
      • 10.0.0.1: Windows 10 / 4C8T / 16 GB DDR4 / JRE 1.8.191
      • 10.0.0.2: Windows 7 / 4C8T / 16 GB DDR3 / JRE 1.8.191

        Attachments

        1. coordinator_log.txt
          106 kB
          Nathan Jackels
        2. non_coordinator_log.txt
          23 kB
          Nathan Jackels
        3. repair_coordinator_console.txt
          32 kB
          Nathan Jackels

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Nathan.Jackels Nathan Jackels
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: