Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21607

Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean)

    XMLWordPrintableJSON

Details

    • Request
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.4.0
    • SQL
    • None

    Description

      AS SPARK-19765, from version 2.1.1, UNCACHE TABLE also un-cache all cached plans that refer to this table.

      My scene like this picture below. When I have finished step 1, I cannot do dropTempView "ods_table1" and "dwd_table2", util I also finish step 2. It waste a lot of memory.

      Can dropTempView function add a param like dropTempView(viewName: String, dropSelfOnly: Boolean). As my scene, After dropTempView "dwd_table2", "dw_table3" would still keep cache in memory.

      Or add a new feacture: supports create materialized views.

      Attachments

        1. spark stream scene.png
          30 kB
          ant_nebula

        Issue Links

          Blocked

          Bug - A problem which impairs or prevents the functions of the product. SPARK-19765 UNCACHE TABLE should also un-cache all cached plans that refer to this table

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21579 dropTempView has a critical BUG

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              maryannxue Wei Xue
              ant_nebula ant_nebula
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: