Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22067

ArrowWriter StringWriter not using position of ByteBuffer holding data

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None

      Description

      When ArrowWriter is copying a StringType column to ArrowData, then StringWriter gets data as a ByteBuffer and sets the ArrowBuf using a position of 0. This currently works because of a bug in Arrow ARROW-1443 that was not making use of the position. This has been fixed as of Arrow 0.7.0 and this surfaced when testing out this version. To fix this, the position of the ByteBuffer should be passed into setSafe instead of 0.

        Attachments

          Activity

            People

            • Assignee:
              bryanc Bryan Cutler
              Reporter:
              bryanc Bryan Cutler
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: