Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24870

Cache can't work normally if there are case letters in SQL

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.1
    • 2.4.0
    • SQL
    • None

    Description

      Cache can't work normally if there are case letters in SQL,
      for example:
      sql("CREATE TABLE IF NOT EXISTS src (key INT, value STRING) USING hive")

      sql("select key, sum(case when Key > 0 then 1 else 0 end) as positiveNum " +
      "from src group by key").cache().createOrReplaceTempView("src_cache")
      sql(
      s"""select a.key
      from
      (select key from src_cache where positiveNum = 1)a
      left join
      (select key from src_cache )b
      on a.key=b.key
      """).explain

       

      The subquery "select key from src_cache where positiveNum = 1" on the left of join can use the cache data, but the subquery "select key from src_cache" on the right of join cannot use the cache data.

      Attachments

        Activity

          People

            eaton eaton
            eaton eaton
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: