Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8598

[Rust] simd_compare_op creates buffer of incorrect length when item count is not a multiple of T::lanes()

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.17.0
    • 1.0.0
    • Rust

    Description

      the simd_compare_op function defined here https://github.com/apache/arrow/blob/master/rust/arrow/src/compute/kernels/comparison.rs#L229

      appears to only work correctly when item count is a multiple of T::lanes().

      Otherwise the resulting boolean array is created with a buffer of incorrect length and subsequent binary operations such as compute::and return an error.

      The no_simd_compare_op function defined in the same module does not appear to have this problem.

      This bug can be reproduced with the following code:

       

      fn main() {
          let lanes = Int8Type::lanes();
          println("i8 lanes: {}", lanes); // 64
          // let item_count = 128; // this works because item_count divides by 64 (lanes) without remainder
          let item_count = 130; // this fails because item_count divides by 64 (lanes) with remainder
          // item_count = 130 should return error:
          // ComputeError("Buffers must be the same size to apply Bitwise AND.")     
          // create boolean array
          let mut select_mask: BooleanArray = vec![true; item_count].into();
          // create arrays with i8 values
          let value_array: PrimitiveArray<Int8Type> = vec![1; item_count].into();
          let filter_array: PrimitiveArray<Int8Type> = vec![2; item_count].into();
          // compare i8 arrays and produce a boolean array
          let result_mask = compute::gt_eq(&value_array, &filter_array).unwrap();
          // compare boolean arrays
           select_mask = compute::and(&select_mask, &result_mask).unwrap();
          // print result, should be all false
          println!("select mask: {:?}", select_mask);
      }
      

       

       

      Attachments

        Issue Links

          Activity

            People

              paddyhoran Paddy Horan
              yordan-pavlov Yordan Pavlov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h