Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-17907

[Website] Blog about Arrow <--> Parquet translation and nesting

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • Website
    • None

    Description

      @tustvold has spent a significant amount of time fixing the Rust implementation of the parquet <–> arrow conversion logic for all the corner cases of nulls, etc. 

       

      During that process, he observed there was a relative lack of information on the topic to be found, so we would like to write some blog posts to remedy that and explain the format and parquet

       

      The basic outline is:

      Part 1: Intro / Encoding Primitive Arrays in Arrow and Parquet
      Part 2: Encoding Structs and Lists  in Arrow and Parquet
      Part 3: Encoding Arbitrary Structs of Lists, Lists of Structs in Arrow and Parquet 

      Attachments

        Activity

          People

            alamb Andrew Lamb
            alamb Andrew Lamb
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10h 20m
                10h 20m