Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33537 Hive Metastore filter pushdown improvement
  3. SPARK-33584

Partition predicate pushdown into Hive metastore support cast string type to date type

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      spark.sql("create table t1(id string) partitioned by (part string) stored as parquet")
      spark.sql("insert into t1 values('1', '2019-01-01')")
      spark.sql("insert into t1 values('2', '2019-01-02')")
      spark.sql("select * from t1 where  part = date '2019-01-01' ").show
      

      We can pushdown {{cast(part as date) = date '2019-01-01' }} to Hive metastore.

        Attachments

          Activity

            People

            • Assignee:
              yumwang Yuming Wang
              Reporter:
              yumwang Yuming Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: