When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo build
Create a new repo, add `arrow=1.0.0` as a dependency, and then run `cargo build`
Expected behavior: After the first successful invocation of `cargo build`, arrow-flight will not recompile if no other changes are made.
Actual behavior: After every invocation of `cargo build`, arrow-flight is recompiled, even when nothing has changed
Create a new crate
Add arrow as a dependency in Cargo.toml:
Now, all invocations of `cargo build` will rebuild arrow, even though nothing in the code has changed:
You can see what is happening by checking out a fresh copy of arrow/master (no Cargo.log) and running `cargo build` – you'll see your local checkout has changes in rust/arrow-flight/src/arrow.flight.protocol.rs:
- Root Cause Analysis
The issue is that the build.rs of arrow-flight calls `tonic_build` to auto generate `rust/arrow-flight/src/arrow.flight.protocol.rs`, which is also checked in (first done in https://github.com/apache/arrow/commit/ec84b7b8102f227295f865c420496830c66a6281).
This file and the version of tonic were updated on https://github.com/apache/arrow/commit/7b49cbc23f22ed99eebf85cc0b9acb1f0d3f832f on July 11, 2020
It turns out that the output of "tonic_build" depends on not only on the version of tonic, but also on the version of proc-macro2, and the version of proc-macro2 is not specifically pinned.
`proc-macro2 = "1.0.19"` was released on July 19, 2020 (https://crates.io/crates/proc-macro2/1.0.19) and it appears to subtlety changes the resulting output of arrow.flight.protocol.rs; Thus the output no longer matches what is checked in. This means that anyone without a Cargo.lock file that pins proc-macro2 to 1.0.18 would get 1.0.19 and thus also a local modification during build.
If we pin Cargo.toml to use proc-macro2 1.0.18 the local modification stops.