Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24870

Cache can't work normally if there are case letters in SQL

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.4.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Cache can't work normally if there are case letters in SQL,
      for example:
      sql("CREATE TABLE IF NOT EXISTS src (key INT, value STRING) USING hive")

      sql("select key, sum(case when Key > 0 then 1 else 0 end) as positiveNum " +
      "from src group by key").cache().createOrReplaceTempView("src_cache")
      sql(
      s"""select a.key
      from
      (select key from src_cache where positiveNum = 1)a
      left join
      (select key from src_cache )b
      on a.key=b.key
      """).explain

       

      The subquery "select key from src_cache where positiveNum = 1" on the left of join can use the cache data, but the subquery "select key from src_cache" on the right of join cannot use the cache data.

        Attachments

          Activity

            People

            • Assignee:
              eaton eaton
              Reporter:
              eaton eaton
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: