Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-10642

[R] Can't get Table from RecordBatchReader with 0 batches

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 3.0.0
    • Component/s: R
    • Environment:

      Description

      Objective is to build a 0 rows data.frame using an arrow schema field definition

       

       

       

      #IPC stream containing only a schema
      stream<-as.raw(c(255,255,255,255,16,1,0,0,16,0,0,0,0,0,10,0,12,0,6,0,5,0,8,0,10,0,0,0,0,1,3,0,12,0,0,0,8,0,8,0,0,0,4,0,8,0,0,0,4,0,0,0,4,0,0,0,160,0,0,0,92,0,0,0,48,0,0,0,4,0,0,0,128,255,255,255,0,0,1,5,20,0,0,0,12,0,0,0,4,0,0,0,0,0,0,0,176,255,255,255,7,0,0,0,82,69,80,79,78,83,69,0,168,255,255,255,0,0,1,5,20,0,0,0,12,0,0,0,4,0,0,0,0,0,0,0,216,255,255,255,6,0,0,0,68,69,84,65,73,76,0,0,208,255,255,255,0,0,1,5,24,0,0,0,16,0,0,0,4,0,0,0,0,0,0,0,4,0,4,0,4,0,0,0,8,0,0,0,68,65,84,65,84,89,80,69,0,0,0,0,16,0,20,0,8,0,6,0,7,0,12,0,0,0,16,0,16,0,0,0,0,0,1,7,36,0,0,0,20,0,0,0,4,0,0,0,0,0,0,0,8,0,12,0,4,0,8,0,8,0,0,0,38,0,0,0,9,0,0,0,8,0,0,0,77,65,67,84,65,95,73,68,0,0,0,0,0,0,0,0))
      readr <- RecordBatchStreamReader$create(stream)
      readr$read_table()
      # Error in Table__from_RecordBatchStreamReader(self) : 
      # Invalid: Must pass at least one record batch or an explicit Schema
      # Now trying to be too clever
      tb <- Table$create(data.frame(), schema = readr$schema)
      dtf <- as.data.frame(tb)
      # This will crash you R session
      

       

       

      Tested on nightly, same behavior. It's borderline a bug / feature request, but to be a drop in replacement for some DBI methods, it needs to be able to build 0 rows data.frame with the correct class for each column.

       

      Thank you and have a nice day.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                npr Neal Richardson
                Reporter:
                meztez Bruno Tremblay
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h