Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6824

Drill Query on MapRDB JSON table failing on schema SchemaChangeException, the only distinct Values are NULL and Text

    Details

      Description

      Drill Query on MapR DB JSON Table or it View fails with a SchemaChangeException. The only distinct values is NULL and some Text.

      The issue is that when Drill sees a NULL or does not see any values for a column in the first batch it assumes the type as NULLABLE INT. Subsequently, when the data shows up and it is different from NULLABLE INT there is a schema change. Operators e.g. aggregators etc. cannot handle such a Schema Change and throw a SchemaChangeException.

       

      One of the short-term solution implemented in this fix: Add a CAST expression which will cast null values to the target type. Hence, we would never see a SchemaChange due to NULLs. However, the MapRDB Reader code was written differently than other reader and was hitting a SchemaChangeException. The code was changed to make it similar to other scans i.e. emit a new batch whenever a schema change is encountered.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gparai Gautam Parai
                Reporter:
                gparai Gautam Parai
                Reviewer:
                Aman Sinha
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 24h
                  24h
                  Remaining:
                  Remaining Estimate - 24h
                  24h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified