Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6971

[Rust] Replace "RecordBatchReader" with "BatchIterator"

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Not A Bug
    • Affects Version/s: 0.15.0
    • Fix Version/s: None
    • Component/s: Rust, Rust - DataFusion
    • Labels:
      None

      Description

      As part of the recent reader work we introduced 

      // arrow::record_batch::RecordBatchReader

      but in datafusion we have

      // datafusion::physical_plan::BatchIterator
      

      These two trait are almost identical (BatchIterator implements Send + Sync whereas RecordBatchReader does not).  I propose we replace RecordBatchReader with BatchIterator (i.e. move it to arrow as it's generally useful outside of datafusion) and update parquet and data fusion accordingly.

      Andy Grove Renjie Liu do you see any issues with this? 

       

       

       

        Attachments

          Activity

            People

            • Assignee:
              paddyhoran Paddy Horan
              Reporter:
              paddyhoran Paddy Horan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: