Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Now that Parquet 1.10 is out, we should consider moving to it, if for no other reason than to address SPARK-23852. 

      There are some non-trivial changes to address. In particular, the ValuesReader interface has changed from passing data as byte[] to a ByteBufferInputStream abstraction that can hide multiple {{ByteBuffer}}s under the hood.

        Attachments

          Activity

            People

            • Assignee:
              rdblue Ryan Blue
              Reporter:
              henryr Henry Robinson
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: