Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22722

Test Coverage for Type Coercion Compatibility

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None
    • Target Version/s:

      Description

      Hive compatibility is pretty important for the users who run or migrate both Hive and Spark SQL.

      We plan to add a SQLConf for type coercion compatibility (spark.sql.typeCoercion.mode). Users can choose Spark's native mode (default) or Hive mode (hive).

      Before we deliver the Hive compatibility mode, we plan to write a set of test cases that can be easily run in both Spark and Hive sides. We can easily compare whether they are the same or not. When new typeCoercion rules are added, we also can easily track the changes. These test cases can also be backported to the previous Spark versions for determining the changes we made.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                yumwang Yuming Wang
                Reporter:
                smilegator Xiao Li
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: