Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7305

Multiple operators do not handle empty batches

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.16.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      While testing the new "EVF" framework, it was found that multiple operators incorrectly handle empty batches. The EVF framework is set up to return a "fast schema" empty batch with only schema as its first batch. It turns out that many operators fail with problems such as:

      • Failure to set the value counts in the output container
      • Fail to initialize the offset vector position 0 to 0 for variable-width or repeated vectors

      And so on.

      Partial fixes are in the JSON reader PR.

      For now, the easiest work-around is to disable the "fast schema" path in the EVF: DRILL-7306.

      To discover the remaining issues, enable the ScanOrchestratorBuilder.enableSchemaBatch option and run unit tests. You can use the VectorChecker and VectorAccessorUtilities.verify() methods to check state. Insert a call to verify() in each "next" method: verify the incoming and outgoing batches. The checker only verifies a few vector types; but these are enough to show many problems.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Paul.Rogers Paul Rogers
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: