Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3720

Avro Record Reader should process Avro files by per block basis

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: Storage - Other
    • Labels:
      None
    • Flags:
      Patch

      Description

      Currently Avro Record Reader processes entire file in single drill bit. This is fine as long as files are of size HDFS block size. However, if the size of the file is large, it causes a lot of performance issues.

      To address this, Avro Record Reader should process Avro files by per block basis.

        Activity

        Hide
        kam_iitkgp Bhallamudi Venkata Siva Kamesh added a comment -

        Attached patch in review board. Review request : 37885

        Show
        kam_iitkgp Bhallamudi Venkata Siva Kamesh added a comment - Attached patch in review board. Review request : 37885
        Hide
        jnadeau Jacques Nadeau added a comment -

        Merged in 8f4ca6e

        Show
        jnadeau Jacques Nadeau added a comment - Merged in 8f4ca6e

          People

          • Assignee:
            jnadeau Jacques Nadeau
            Reporter:
            kam_iitkgp Bhallamudi Venkata Siva Kamesh
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development