[NIFI-3593] Conditionally Ignore Junit Tests

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Information Provided
• Affects Version/s: 1.0.0, 1.2.0
• Component/s: Tools and Build
• Labels: build environment junit
• Assignee: Unassigned
• Reporter: Puspendu Banerjee
• Created: 13/Mar/17 22:26
• Updated: 25/Mar/22 16:31
• Resolved: 25/Mar/22 16:31

Description

We should incorporate a capability to conditionally ignore unit tests which are not relevant to certain condition, environment, language, locale etc.
For example,

1. YUI compressor fails on windows
2. CEFParser fails on non-English environment and we are yet to get an updated version for that.

Definitely, we can craft our test-cases to fit in, but that may lead to unintended foul-play. Instead of that if we run those tests in a quarantine, still we will have a clear view of what ran and what not as well as we will be able to proceed through a green build.

As long as release management is concerned,

1. we need to be very careful about choosing what to quarantine or conditionally ignore
2. we should have a clean quarantine before release for certain environment.
   I am thinking about some annotation driven solution like the below one, so that we can easily track it and report.

   @IgnoreOn(since:Date, reason:String, conditionExpr:String)
   

The way Junit's assume or assumeThat works doesn't fit in directly.
Looking forward for inputs.

Ref: https://martinfowler.com/articles/nonDeterminism.html#Quarantine

Issue Links

• is superceded by: NIFI-9084 Refactor unit and integration tests to use JUnit 5 instead of JUnit 4
• relates to: NIFI-3666 Skipped tests on windows need to be validated or fixed