Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-3261

s3 byte range for just schema or row count

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.11.0
    • None
    • community, misc, python, tools
    • None

    Description

      boto3 python library has s3 get_object that accepts a byte range (to avoid downloading a whole file but just download selected byte range). if my Avro file is 100mb, can Avro library do some byte range seek to only download the s3 object partially (ie the parts that contain header.schema metadata and row count)?

      Attachments

        Activity

          People

            Unassigned Unassigned
            toopt4 t oo
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: