Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-847

Merging Receiver requires to canonicalize schemas of input batches

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      The Merge Receivers assumes all the incoming batches have the same schemas. If the incoming batches happen to have different schemas, the run-time generated code for merge receiver would not be able to handle the different schemas, and could throw ClassCast Exception.

      To fix it, we need canonicalize the schemas of incoming batches for merge receiver. If the schemas are still different after this step, Merge Receiver would throw exception.

        Issue Links

          Activity

          Hide
          jnadeau Jacques Nadeau added a comment -

          Fixed in 602f817

          Show
          jnadeau Jacques Nadeau added a comment - Fixed in 602f817
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user vdiravka opened a pull request:

          https://github.com/apache/drill/pull/1017

          The query with "SELECT *" with "ORDER BY" clause and `planner.slice_t…

          …arget`=1 doesn't preserve column order

          Issue: Columns ordering doesn't preserve for the star query with sorting when this is planned into multiple fragments.
          Solution: The commit for DRILL-847 is oudated. There is no need to canonicalize the batch or container since RecordBatchLoader swallows the "schema change" for now if two batches have different column ordering.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/vdiravka/drill DRILL-5822

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/drill/pull/1017.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1017


          commit 922505c35ed18f5cf76a5ded77a734827c15eb59
          Author: Vitalii Diravka <vitalii.diravka@gmail.com>
          Date: 2017-10-26T18:07:33Z

          The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order

          • The commit for DRILL-847 is oudated. There is no need to canonicalize the batch or container since RecordBatchLoader
            swallows the "schema change" for now if two batches have different column ordering.

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user vdiravka opened a pull request: https://github.com/apache/drill/pull/1017 The query with "SELECT *" with "ORDER BY" clause and `planner.slice_t… …arget`=1 doesn't preserve column order Issue: Columns ordering doesn't preserve for the star query with sorting when this is planned into multiple fragments. Solution: The commit for DRILL-847 is oudated. There is no need to canonicalize the batch or container since RecordBatchLoader swallows the "schema change" for now if two batches have different column ordering. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vdiravka/drill DRILL-5822 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/1017.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1017 commit 922505c35ed18f5cf76a5ded77a734827c15eb59 Author: Vitalii Diravka <vitalii.diravka@gmail.com> Date: 2017-10-26T18:07:33Z The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order The commit for DRILL-847 is oudated. There is no need to canonicalize the batch or container since RecordBatchLoader swallows the "schema change" for now if two batches have different column ordering.

            People

            • Assignee:
              jni Jinfeng Ni
              Reporter:
              jni Jinfeng Ni
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development