Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20358

Allow setting variable value from Hive metastore table properties

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Hive already supports set command as well as variable substitution:

      set start_ds=2018-08-01;

      SELECT COUNT( * ) FROM t WHERE ds >= '${hiveconf:start_ds}';

       

      Or:

      set start_ds='2018-08-01';

      SELECT COUNT( * ) FROM t WHERE ds >= ${hiveconf:start_ds};

       

      This issue propose to extend the set syntax to allow running UDF and a UDF that queries metastore:

      SET <VARIABLE_NAME> <EXPRESSION>;

       

      For example:

      set start_ds GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');

      set start_ds GET_PARTITION_PROPERTY('mydb', 'mytable', 'ds=2018-01-01/hr=12', 'last_modified_time');

       

      This will allow query workflows like the following:

      set last_run_time GET_TABLE_PROPERTY('mydb', 'mytable', 'last_modified_time');

      INSERT INTO TABLE mytable SELECT * FROM src WHERE src.time > last_run_time;

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zshao Zheng Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: