Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18909

The error message in `ExpressionEncoder.toRow` and `fromRow` is too verbose

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.2.0
    • SQL
    • None

    Description

      In `ExpressionEncoder.toRow` and `fromRow`, we will catch the exception and put the treeString of serializer/deserializer expressions in the error message. However, encoder can be very complex and the serializer/deserializer expressions can be very large trees and blow up the log files(e.g. generate over 500mb logs for this single error message.)

      We should simplify it.

      Attachments

        Activity

          People

            dongjoon Dongjoon Hyun
            cloud_fan Wenchen Fan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: