Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10153 Support time travel for Iceberg tables
  3. IMPALA-10288

Functionality to display table history for Iceberg tables

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.0.0
    • Frontend
    • None
    • ghx-label-8

    Description

      For time travel it would be useful to display the table history (previous snapshot, creation times etc.) so that the user can pick which snapshot to use.

      Spark already has this functionality:
      https://iceberg.apache.org/spark/#time-travel

      SELECT * FROM prod.db.table.history
      
      +-------------------------+---------------------+---------------------+---------------------+
      | made_current_at         | snapshot_id         | parent_id           | is_current_ancestor |
      +-------------------------+---------------------+---------------------+---------------------+
      | 2019-02-08 03:29:51.215 | 5781947118336215154 | NULL                | true                |
      | 2019-02-08 03:47:55.948 | 5179299526185056830 | 5781947118336215154 | true                |
      | 2019-02-09 16:24:30.13  | 296410040247533544  | 5179299526185056830 | false               |
      | 2019-02-09 16:32:47.336 | 2999875608062437330 | 5179299526185056830 | true                |
      | 2019-02-09 19:42:03.919 | 8924558786060583479 | 2999875608062437330 | true                |
      | 2019-02-09 19:49:16.343 | 6536733823181975045 | 8924558786060583479 | true                |
      +-------------------------+---------------------+---------------------+---------------------+
      

      We can have the same output columns for the Impala implementation, however, the actual syntax to run the query is not decided yet.

      There are various solutions on the field:
      1) SELECT * from tablename.history; (Spark)
      2) DESCRIBE HISTORY tablename; (Delta)
      3) select * from 'table.history'; (Presto)

      Attachments

        Activity

          People

            gaborkaszab Gabor Kaszab
            gaborkaszab Gabor Kaszab
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: