Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6543

Limit RowBatch serialization size to INT_MAX

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.11.0, Impala 2.12.0
    • Impala 2.12.0
    • Backend
    • None

    Description

      A RowBatch does not support serializing an uncompressed size of greater than INT_MAX. The serialization format relies on tuple offsets of type of int32, so they cannot address greater than INT_MAX.

      On debug builds, a DCHECK in SerializeInternal() fires when the size exceeds INT_MAX. On release builds, LZ4's limitation of INT_MAX on the size of uncompressed size avoids any issue. The serialization codepath should protect itself and avoid this issue on debug builds.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            joemcdonnell Joe McDonnell
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment