Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8947

Improve expression type coercion, casting & checking

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • SQL
    • None

    Description

      This is an umbrella ticket to improve type casting & checking.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-2863 Emulate Hive type coercion in native reimplementations of Hive functions

          • Major - Major loss of function.
          • Closed
          1.
          		 Support AbstractDataType in TypeCollection Sub-task Resolved Reynold Xin
          2.
          		 Support TypeCollection in ExpectsInputTypes Sub-task Resolved Reynold Xin
          3.
          		 Move expression specific type coercion into expression themselves Sub-task Resolved Unassigned
          4.
          		 Throw type mismatch in check analysis for expressions with expected input types defined Sub-task Resolved Michael Armbrust
          5.
          		 Implement implicit type cast for expressions that define expected input types Sub-task Resolved Reynold Xin
          6.
          		 Add ExpectsInputTypes trait for defining expected input types. Sub-task Resolved Reynold Xin
          7.
          		 Rename ExpectsInputTypes => AutoCastInputTypes Sub-task Resolved Reynold Xin
          8.
          		 Add constraint for Children expression for data type Sub-task Closed Reynold Xin
          9.
          		 clean up type checking in math.scala Sub-task Resolved Reynold Xin
          10.
          		 Add ExpectsInputTypes to expressions whenever possible Sub-task Resolved Reynold Xin
          11.
          		 Improve error reporting for expression data type mismatch Sub-task Resolved Wenchen Fan
          12.
          		 apply type checking interface to more expressions Sub-task Resolved Wenchen Fan
          13.
          		 do type coercion for key and when expressions in CaseKeyWhen Sub-task Resolved Wenchen Fan
          14.
          		 equality check between boolean type and numeric type is broken. Sub-task Resolved Wenchen Fan
          15.
          		 move type-check from BinaryArithmetic and BinaryComparison to BinaryOperator Sub-task Closed Reynold Xin
          16.
          		 More comprehensive type checking in expressions Sub-task Resolved Reynold Xin
          17.
          		 BinaryOperator expression Sub-task Resolved Reynold Xin
          18.
          		 WidenTypes should also support Intersect and Except in addition to Union Sub-task Resolved Yijie Shen
          19.
          		 Fix and re-enable ignored tests in ExpressionTypeCheckingSuite Sub-task Resolved Wenchen Fan
          20.
          		 refactor the implicit type cast code Sub-task Resolved Wenchen Fan
          21.
          		 DecimalPrecision should also support Intersect and Except in addition to Union Sub-task Resolved L. C. Hsieh
          22.
          		 UnionPushDown should also support Intersect and Except in addition to Union Sub-task Resolved Yijie Shen
          23.
          		 Don't implicitly cast non-atomic types to string type Sub-task Resolved Reynold Xin
          24.
          		 Analysis should check that join conditions' data types are booleans Sub-task Resolved Josh Rosen
          25.
          		 Analysis should detect sorting on unsupported column types Sub-task Resolved Josh Rosen
          26.
          		 SortMergeJoin crashes when trying to join on unsortable columns Sub-task Resolved L. C. Hsieh
          27.
          		 Centralize orderable data type checking Sub-task Resolved Reynold Xin

          Activity

            People

              rxin Reynold Xin
              rxin Reynold Xin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: