Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-14665

[Java] JdbcToArrowUtils ResultSet iteration bug

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 6.0.0
    • 8.0.0
    • Java

    Description

      When specifying a target batch size, the iteration logic is currently broken:

              while (rs.next() && readRowCount < config.getTargetBatchSize()) {
                compositeConsumer.consume(rs);
                readRowCount++;
              }
      
      

      calling next() on the result set will move the cursor forward to the next row, even when we've reached the target batch size.

      For example, consider setting target batch size to 1, and query a table that has three rows.

      On the first iteration, we'll successfully consume the first row. On the next iteration, we'll move the cursor to row 2, but detect the read row count is no longer < target batch size and return.

      Upon calling into the method again with the same result set, rs.next will be called again which will result in successfully consuming row 3.

      Problem: row 2 is skipped!

      Attachments

        Issue Links

          Activity

            People

              zacMode Zac
              zacMode Zac
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h