Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4010

In HBase reader, create child vectors for referenced HBase columns to avoid spurious schema changes

    XMLWordPrintableJSON

    Details

      Description

      HBaseRecordReader needs to create child vectors for all referenced/requested columns.

      Currently, if a fragment reads only HBase rows that don't have a particular referenced column (within a given column family), downstream code adds a dummy column of type NullableIntVector (as a child in the MapVector for the containing HBase column family).

      If any other fragment reads an HBase row that does contain the referenced column, that fragment's reader will create a child NullableVarBinaryVector for the referenced column.

      When the data from those two fragments comes together, Drill detects a schema change, even though logically there isn't really any schema change.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dsbos Daniel Barclay
                Reporter:
                dsbos Daniel Barclay
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: