[ARROW-12145] [Developer][Archery] Flaky test: test_static_runner_from_json

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Component/s: Developer Tools
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27967
• Assignee: Diana Clarke
• Reporter: Diana Clarke
• Created: 29/Mar/21 22:36
• Updated: 11/Jan/23 08:24
• Resolved: 31/Mar/21 01:38

Description

This test assumes:

 artificial_reg, normal = RunnerComparator(contender, baseline).comparisons

When the return order could be:

 normal, artificial_reg = RunnerComparator(contender, baseline).comparisons

The return order of comparisons isn't deterministic.

Issue Links

• is duplicated by: ARROW-12149 [Dev] Archery benchmark test case is failing
• links to: GitHub Pull Request #9843