[KAFKA-4450] Add missing 0.10.1.x upgrade tests and ensure ongoing compatibility checks

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.10.1.0
• Fix Version/s: 0.10.2.0
• Component/s: system tests
• Assignee: Ewen Cheslack-Postava
• Reporter: Ewen Cheslack-Postava
• Reviewer: Ismael Juma
• Created: 27/Nov/16 05:12
• Updated: 28/Jan/17 01:44
• Resolved: 28/Jan/17 01:44

Description

We have upgrade system tests, but we neglected to update them for the most recent released versions (we only have LATEST_0_10_0 but not something from 0_10_1).

We should probably not only add these versions, but also a) make sure some TRUNK version is always included since upgrade to trunk would always be possible to avoid issues for anyone deploying off trunk (we want every commit to trunk to be solid & compatible) and b) make sure there aren't gaps between versions annotated on the test vs versions that are officially released (which may not be easy statically with the decorators, but might be possible by checking the kafkatest version against previous versions and checking for gaps?).

Perhaps we need to be able to get the most recent release/snapshot version from the python code so we can always validate previous versions? Even if that's possible, is there going to be a reliable way to get all the previous released versions so we can make sure we have all upgrade tests in place?

Issue Links

• links to: GitHub Pull Request #2457