Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20537

OffHeapColumnVector reallocation may not copy existing data

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.0
    • 2.2.0
    • Spark Core
    • None

    Description

      As SPARK-20474 revealed, reallocation in OnHeapColumnVector may copy a part of the original storage.
      OffHeapColumnVector reallocation also copies to the new storage data up to elementsAppended. This variable is only updated when using the ColumnVector.appendX API, while ColumnVector.putX is more commonly used.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kiszk Kazuaki Ishizaki
            kiszk Kazuaki Ishizaki
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment