[CASSANDRA-16211] Improve job metadata queries exception handling in cassandra-diff

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0.1
• Component/s: Tool/diff
• Labels: pull-request-available
• Change Category: Operability
• Complexity: Low Hanging Fruit
• Platform: All

Source Control Link

https://github.com/apache/cassandra-diff/commit/e3161b60b78b5cfbeba4875acbbb2f138d5da051

Test and Documentation Plan

unit test

• Assignee: Yifan Cai
• Reporter: Yifan Cai

Authors

Yifan Cai

Reviewers

Dinesh Joshi

• Created: 14/Oct/20 20:08
• Updated: 01/Aug/21 11:33
• Resolved: 11/May/21 22:33

Description

The job metadata tracks the progress of the diff job. Sometimes, a job can fail due to the progress update query failures.
The progress update queries can be categorized into 2 groups, critical and trivial one.
When a query failed to update a trivial status (e.g. ProgressTracker), we would mostly hope to continue the job and just log the failure.
When a query failed to update a critical status (e.g. JobLifeCycle), we can apply the client-side retry strategy (e.g. exponential backoff) in addition to the retry policy.

Issue Links

• relates to: CASSANDRA-15537 4.0 quality testing: Local Read/Write Path: Upgrade and Diff Test
• links to: GitHub Pull Request #15