[ARROW-12581] [C++][FlightRPC] Benchmark compression with real data

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.0.0
• Component/s: C++, FlightRPC
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/28337
• Assignee: David Li
• Reporter: yibocai#1
• Created: 28/Apr/21 05:07
• Updated: 11/Jan/23 08:27
• Resolved: 01/May/21 14:26

Description

Flight benchmark program is using random test data as payload. Random data is hard to compress. Per my test, the compressed payload (zstd default level) is actually a bit larger than the original uncompressed payload. So compression is a pure loss in current benchmark.

It's better to use real world test set for a more reasonable benchmark about how compression influences FlightRPC performance. Perhaps the benchmark (both client/server) could accept a path to an IPC file as an option.

Issue Links

• links to: GitHub Pull Request #10212