Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5291

Parquet Reader produces low density batches - variable width fields

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: None
    • Labels:
      None

      Description

      See DRILL-5266 for background. That JIRA analyzed the problem with Parquet producing "low density" record batches. That JIRA focused on the issue with fixed-width fields: due to a bug, we overestimated the space taken.

      Once that bug is fixed, Parquet continues to produce low density batches for variable-width fields. DRILL-5266 explains why.

      This ticket covers the variable-width case so that we don't lose sight of it once the fixed-width case is fixed.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              paul-rogers Paul Rogers
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: