Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7313

Use Hive schema for MaprDB native reader when field was empty and support all text mode

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.16.0
    • Fix Version/s: 1.17.0
    • Component/s: None

      Description

      Currently, when an external Hive MaprDB table is queried using hive plugin with enabled store.hive.maprdb_json.optimize_scan_with_native_reader, some queries may fail due to soft schema change, though Hive knows actual data types.

      For example, when we have a table with several fields, and one of them has only several non-null values, queries with grouping by such field will fail due to schema change.

      The goal of this Jira is to allow using types from Hive when a non-existing field is created, so it will allow avoiding such issues.

      In the scope of this Jira is also added new session option store.hive.maprdb_json.all_text_mode to read all data from the maprDB JSON tables as VARCHAR when hive plugin is used and Drill native MaprDB JSON reader usage is enabled.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                volodymyr Vova Vysotskyi
                Reporter:
                volodymyr Vova Vysotskyi
                Reviewer:
                Arina Ielchiieva
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: