Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-11632

[Rust] csv::Reader doesn't propagate schema metadata to RecordBatches

    XMLWordPrintableJSON

Details

    Description

      Rust csv::Reader doesn't propagate schema metadata to RecordBatches it creates:

       

      let reader = csv::ReaderBuilder::new()
          .with_schema(Arc::new(Schema::new_with_metadata(fields, metadata)))
          .build(file)
          .unwrap();
      
      let batch = reader.next().unwrap().unwrap();
      
      assert_eq!(reader.schema(), batch.schema());

       

      The parquet writer crate complains about the schema non matching with this error:
      Record batch schema does not match writer schema
       

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ithkuil Marko Mikulicic
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m