Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5396

A flatten query on top of 2 files with one record each causes oversize allocation error randomly

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.10.0, 1.11.0
    • None
    • None

    Description

      git.commit.id.abbrev=38ef562

      As part of verifying DRILL-3562, I came up with the below 2 files

      File 1:

      { "a": { "b": { "c": [] } } }
      

      File 2:

      { "a": { "b": { "c": [1] } } }
      

      Now the below query work on individual files, however when I run the query on a directory containing both the files, I randomly hit the below error

      select FLATTEN(t.a.b.c) AS c from dfs.`/drill/testdata/json_kvgenflatten/drill3562` t;
      Error: SYSTEM ERROR: OversizedAllocationException: Unable to expand the buffer. Max allowed buffer size is reached.
      
      Fragment 0:0
      
      [Error Id: e556243b-7ad6-4131-81f7-e8b225c9c8bc on qa-node182.qa.lab:31010] (state=,code=0)
      

      This could be related to the fix for DRILL-3562

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            rkins Rahul Kumar Challapalli

            Dates

              Created:
              Updated:

              Slack

                Issue deployment