Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11160 Auto-gather column stats
  3. HIVE-16957

Support CTAS for auto gather column stats

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • None
    • None

    Description

      The idea is to rely as much as possible on the logic in ColumnStatsSemanticAnalyzer as other operations do. In particular, they create a 'analyze table t compute statistics for columns', use ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE MATERIALIZED VIEW is that the table object does not exist yet, hence we cannot rely fully on ColumnStatsSemanticAnalyzer.

      Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a statement for column stats collection that uses a table values clause instead of the original table reference:

      select compute_stats(col1), compute_stats(col2), compute_stats(col3)
      from table(values(cast(null as int), cast(null as int), cast(null as string))) as t(col1, col2, col3);
      

      Attachments

        1. HIVE-16957.01.patch
          3.12 MB
          jcamachorodriguez
        2. HIVE-16957.02.patch
          3.32 MB
          jcamachorodriguez
        3. HIVE-16957.03.patch
          3.29 MB
          jcamachorodriguez
        4. HIVE-16957.04.patch
          3.29 MB
          jcamachorodriguez
        5. HIVE-16957.05.patch
          3.30 MB
          jcamachorodriguez
        6. HIVE-16957.06.patch
          3.30 MB
          jcamachorodriguez
        7. HIVE-16957.07.patch
          3.30 MB
          jcamachorodriguez
        8. HIVE-16957.07.patch
          3.30 MB
          jcamachorodriguez
        9. HIVE-16957.08.patch
          3.32 MB
          jcamachorodriguez
        10. HIVE-16957.08.patch
          3.32 MB
          jcamachorodriguez
        11. HIVE-16957.09.patch
          3.32 MB
          jcamachorodriguez
        12. HIVE-16957.09.patch
          3.32 MB
          jcamachorodriguez
        13. HIVE-16957.10.patch
          3.32 MB
          jcamachorodriguez
        14. HIVE-16957.10.patch
          3.32 MB
          jcamachorodriguez
        15. HIVE-16957.10.patch
          3.32 MB
          jcamachorodriguez
        16. HIVE-16957.10.patch
          3.32 MB
          jcamachorodriguez
        17. HIVE-16957.patch
          414 kB
          jcamachorodriguez

        Issue Links

          Activity

            People

              jcamacho Jesús Camacho Rodríguez
              pxiong Pengcheng Xiong
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: