Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1645

Can not query any column if the row size is larger than 128K

    XMLWordPrintableJSON

    Details

      Description

      If the row size is larger than 128K, drill can not query any column no matter how small the column is.
      So once the top level row of the json file is larger than 128K, then we can not query anything from this file.

      We need drill can query small columns which is below the limitation at least.

      Reproduce:

      0: jdbc:drill:> select * from `wrapper.json`;
      Query failed: Failure while running fragment. Record is too long. Max allowed record size is 131072 bytes. [7894614e-9928-4aa5-b649-2d085d01a55b]
      
      Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
      0: jdbc:drill:> select id from `wrapper.json`;
      Query failed: Failure while running fragment. Record is too long. Max allowed record size is 131072 bytes. [ed61693e-d4da-478e-b85d-1fb7d9621243]
      
      Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
      

      The sample data format is like below(Complete sample data is attached.):
      In above test case, the "id" column should be fetched since it is very small.

      [root@maprdemo data]# more wrapper.json 
      {"id": 1,"supercolumn": {"content":testtesttesttesttesttestt......."}}
      

        Attachments

        1. wrapper.json
          144 kB
          Hao Zhu

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                haozhu Hao Zhu
              • Votes:
                1 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: