Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-16085

[C++][R] InMemoryDataset::ReplaceSchema does not alter scan output

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 7.0.0
    • 9.0.0
    • C++, R

    Description

       

      The following fails:

      sub_df1 <- Table$create(
        x = Array$create(c(1, 2, 3)),
        y = Array$create(c("a", "b", "c"))
      )
      sub_df2 <- Table$create(
        x = Array$create(c(4, 5)),
        z = Array$create(c("d", "e"))
      )
      
      ds1 <- InMemoryDataset$create(sub_df1)
      ds2 <- InMemoryDataset$create(sub_df2)
      ds <- c(ds1, ds2)
      actual <- ds %>% collect()
      
      Type error: yielded batch had schema x: double
      y: string which did not match InMemorySource's: x: double
      y: string
      z: string
      /Users/willjones/Documents/arrows/arrow-quick/cpp/src/arrow/util/iterator.h:541  child_.Next()
      /Users/willjones/Documents/arrows/arrow-quick/cpp/src/arrow/util/iterator.h:152  value_.status()
      /Users/willjones/Documents/arrows/arrow-quick/cpp/src/arrow/util/iterator.h:180  maybe_element
      /Users/willjones/Documents/arrows/arrow-quick/cpp/src/arrow/dataset/scanner.cc:840  fragments_it.ToVector()
      

       

      Attachments

        Issue Links

          Activity

            People

              wjones127 Will Jones
              wjones127 Will Jones
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m