Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15250

Reuse partitions info generated in MoveTask to its subscribers (StatsTask)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • Metastore
    • None

    Description

      When dynamic partitions are enabled, StatsTask loads partition information by querying metastore. In cases like insert overwrite table, this can be expensive operation depending on the number of partitions involved (for e.g, in tpcds populating web_returns table would incur 2184 DB calls just on this function).

      It would be good to pass on the partition information generated in MoveTask to its subscribers to reduce the number of DB calls.

      Attachments

        1. HIVE-15250.1.patch
          3 kB
          Rajesh Balamohan
        2. HIVE-15250.2.patch
          3 kB
          Rajesh Balamohan
        3. HIVE-15250.3.patch
          3 kB
          Rajesh Balamohan

        Issue Links

          Activity

            People

              rajesh.balamohan Rajesh Balamohan
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: