Apache Drill
  1. Apache Drill
  2. DRILL-1061

Add or modify Optiq rule such that two projects involving complex JSON objects should not be collapsed into one.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently, Drill could not nest a function whose output is a complex JSON object into another function directly. For instance,

      select foo.mycol.x from (
      select convert_from('

      { x : [1,2], y : 100 }

      ', 'JSON') as mycol
      from cp.`tpch/nation.parquet`) as foo(mycol);

      select convert_to(foo.mycol.x, 'JSON') from (
      select convert_from('

      { x : [1,2], y : 100 }

      ', 'JSON') as mycol
      from cp.`tpch/nation.parquet`) as foo(mycol);

      Optiq will merge the project in the out subquery with the project in the inner subquery into one project expression. This cause issues in Drill side.

      In stead, we should add or modify Optiq rule so that the two projects will stay where they are, and evaluate the convert_from () first and produce a record batch. The second project will read from the record batch, and produce its output.

      Doing that will enable Drill to use out / inner subquery to refer the complex JSON object .

        Activity

        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail, testing [ 12871583 ] Drill workflow [ 12933873 ]
        Jacques Nadeau made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Jacques Nadeau made changes -
        Fix Version/s 0.4.0 [ 12324963 ]
        Mehant Baid made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Mehant Baid [ mehant ] DrillCommitter [ drillcommitter ]
        Mehant Baid made changes -
        Attachment DRILL-1061.patch [ 12656802 ]
        Cliff Buchanan made changes -
        Field Original Value New Value
        Assignee Mehant Baid [ mehant ]
        Jinfeng Ni created issue -

          People

          • Assignee:
            DrillCommitter
            Reporter:
            Jinfeng Ni
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development