[SOLR-13176] Testing of TLOG Replicas needs to be re-instated, may be hiding bugs

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Chris M. Hostetter
• Created: 29/Jan/19 02:18
• Updated: 02/Oct/19 17:30

Description

As part of mark miller's push to cleanup tests, one change he made as part of his big_ SOLR-12801 commit (circa Nov2018) was to dissable the randomized use of TLOG replicas in a lot of tests

His comments at the time were that he suspected a lot of the problems he was seeing was due to a poor implementation of TestInjection.waitForInSyncWithLeader() (which only comes into play for TLOG replicas) ultimately leading to him creating SOLR-12313.

But based on some limited experimentation I made w/trying to re-enable TLOG replica randomization in some tests after (essentially) removing TestInjection.waitForInSyncWithLeader() in SOLR-13168 i'm still seeing a lot of sporadic test failures when TLOG replicas get used... the only change is that instead of "failing slow" because of the stalls introduced by TestInjection.waitForInSyncWithLeader() they started failing quickly.

It's not clear if these failures are because the tests have bugs; or if the tests don't account for the expected behavior of the TLOG replica types in certain situations; or if the code paths being tested have bugs when dealing with TLOG replicas.

---

Bottom line: As things stand today, TLOG replicas aren't being very thoroughly tested, particularly in edge cases (http partitions, LIR, leader election, mixed used of replica types, etc...)

Issue Links

• is a parent of: SOLR-13302 Adding HttpPartitionTest and ForceLeaderTest for TLOG replicas
• relates to: SOLR-13168 tlog replicas wait for sync on every commit when solr is run with java assertions
• relates to: SOLR-12313 Removing TestInjection#waitForInSyncWithLeader