Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21954

JacksonUtils should verify MapType's value type instead of key type

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.1, 2.3.0
    • SQL
    • None

    Description

      JacksonUtils.verifySchema verifies if a data type can be converted to JSON. For MapType, it now verifies the key type. However, in JacksonGenerator, when converting a map to JSON, we only care about its values and create a writer for the values. The keys in a map are treated as strings by calling toString on the keys.

      Thus, we should change JacksonUtils.verifySchema to verify the value type of MapType.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            viirya L. C. Hsieh
            viirya L. C. Hsieh
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment