Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1645

Can not query any column if the row size is larger than 128K

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      If the row size is larger than 128K, drill can not query any column no matter how small the column is.
      So once the top level row of the json file is larger than 128K, then we can not query anything from this file.

      We need drill can query small columns which is below the limitation at least.

      Reproduce:

      0: jdbc:drill:> select * from `wrapper.json`;
      Query failed: Failure while running fragment. Record is too long. Max allowed record size is 131072 bytes. [7894614e-9928-4aa5-b649-2d085d01a55b]
      
      Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
      0: jdbc:drill:> select id from `wrapper.json`;
      Query failed: Failure while running fragment. Record is too long. Max allowed record size is 131072 bytes. [ed61693e-d4da-478e-b85d-1fb7d9621243]
      
      Error: exception while executing query: Failure while trying to get next result batch. (state=,code=0)
      

      The sample data format is like below(Complete sample data is attached.):
      In above test case, the "id" column should be fetched since it is very small.

      [root@maprdemo data]# more wrapper.json 
      {"id": 1,"supercolumn": {"content":testtesttesttesttesttestt......."}}
      

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              haozhu Hao Zhu

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment