Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15928

Throw FSWriteError upon write failures in order to apply DiskFailurePolicy

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 4.0-beta2, 4.0
    • Local/Other
    • None
    • Degradation - Resource Management
    • Low
    • Low Hanging Fruit
    • Code Inspection
    • All
    • None
    • [0e65e85861b84e87691d576ba80c5c3b3b7020d5|https://github.com/apache/cassandra/commit/0e65e85861b84e87691d576ba80c5c3b3b7020d5

    • Hide

      ci

      Show
      ci

    Description

      Active handling of FSError in AbstractLocalAwareExecutorService was introduced in CASSANDRA-14993
       
      However, in the current code base, there are places that catch the IOException but not wrap it with FSError, hence not triggering the DiskFailurePolicy. Especially when the bad disk no longer permits write operations, it could leads to the mutation stage backup. Therefore I propose to fix the IOException handling in those cases. 
       
      From the code inspection, those 6 places are current not re-throwing an IOException with FSWriteError.

      1. org.apache.cassandra.triggers.CustomClassLoader#addClassPath throws IOError. Invoked in TriggerExecutor's constructor
      2. org.apache.cassandra.io.util.FileUtils#renameWithConfirm throws RuntimeException
      3. org.apache.cassandra.io.util.FileUtils#truncate throws RuntimeException
      4. org.apache.cassandra.io.util.FileUtils#write throws RuntimeException
      5. org.apache.cassandra.db.compaction.LeveledManifest#sendBackToL0 throws RuntimeException. Invokes rewriteSSTableMetadata
      6. org.apache.cassandra.io.sstable.SSTableHeaderFix#writeNewMetadata throws RuntimeException. Invokes rewriteSSTableMetadata

      Attachments

        Activity

          People

            yifanc Yifan Cai
            yifanc Yifan Cai
            Yifan Cai
            Aleksey Yeschenko
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h