Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-27260

OOM while retrieving delegation tokens using get_all_token_identifiers call

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Hive, Standalone Metastore
    • None

    Description

      get_all_token_identifiers is retrieving all the entries in table DELEGATION_TOKENS at once.

      In systems where the total number of rows is very high causes OOM in HMS while retrieving all the rows at once.

      Can we have batching kind of mechanism while retrieving rows from that table?

      How about we add timestamp column to that table so that old entries can be cleaned up manually incase going via cleaner did not help?

      Even expiry token removal thread is also using the same api get_all_token_identifiers and it becomes difficult for cleaner once the number of rows reaches higher number

       

      There is a feature https://issues.apache.org/jira/browse/HIVE-17609 but that is also using get_all_token_identifiers, so it will also lead to the same issue

      Attachments

        Activity

          People

            Unassigned Unassigned
            tarak271 Taraka Rama Rao Lethavadla
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: