[ARROW-11027] [Rust] [Parquet] Intermittent test failure thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool'

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/26946
• Assignee: Unassigned
• Reporter: Andrew Lamb
• Created: 24/Dec/20 13:14
• Updated: 11/Jan/23 08:17
• Resolved: 26/Apr/21 12:43

Description

The following parquet tests intermittently fail

test encodings::encoding::tests::test_bool ... FAILED

thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18

ARROW-10943 tracks an intermittent error in the parquet tests. We think we fixed one error but there is still something wrong.

I can reproduce this locally at ca685a0c08bb41f43a80e5605e4cc8f9efb77cca by running the following shell script:

#!/bin/bash
# run a command in a loop

cd /Users/alamb/Software/arrow2/rust
export PARQUET_TEST_DATA=`pwd`/../cpp/submodules/parquet-testing/data
export ARROW_TEST_DATA=`pwd`/../testing/data
export RUST_BACKTRACE=1

i=0
while true; do
    echo "iteration $i"
    cargo test -p parquet  -- test_bool
    i=$((i+1))
done

Here is an example of the full stack trace / errors:

test encodings::encoding::tests::test_bool ... FAILED

failures:

---- encodings::encoding::tests::test_bool stdout ----
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet/src/util/bit_util.rs:73:18
stack backtrace:
   0: std::panicking::begin_panic
             at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/panicking.rs:505
   1: <bool as parquet::util::bit_util::FromBytes>::from_ne_bytes
             at ./src/util/bit_util.rs:73
   2: parquet::util::bit_util::from_ne_slice
             at ./src/util/bit_util.rs:32
   3: parquet::encodings::rle::RleDecoder::get_batch
             at ./src/encodings/rle.rs:402
   4: <parquet::encodings::decoding::RleValueDecoder<T> as parquet::encodings::decoding::Decoder<T>>::get
             at ./src/encodings/decoding.rs:322
   5: parquet::encodings::encoding::tests::put_and_get
             at ./src/encodings/encoding.rs:1271
   6: <T as parquet::encodings::encoding::tests::EncodingTester<T>>::test_internal
             at ./src/encodings/encoding.rs:1214
   7: parquet::encodings::encoding::tests::EncodingTester::test
             at ./src/encodings/encoding.rs:1159
   8: parquet::encodings::encoding::tests::test_bool
             at ./src/encodings/encoding.rs:969
   9: parquet::encodings::encoding::tests::test_bool::{{closure}}
             at ./src/encodings/encoding.rs:966
  10: core::ops::function::FnOnce::call_once
             at /Users/alamb/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
  11: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Issue Links

• relates to: ARROW-10943 [Rust] Intermittent build failure in parquet encoding