Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17861 Store data source partitions in metastore and push partition pruning into metastore
  3. SPARK-18545

Verify number of hive client RPCs in PartitionedTablePerfStatsSuite

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.1.0
    • SQL
    • None

    Description

      To avoid performance regressions like https://issues.apache.org/jira/browse/SPARK-18507 in the future, we should add a metric for the number of Hive client RPC issued and check it in the perf stats suite.

      cc cloud_fan

      Attachments

        Issue Links

          relates to

          Sub-task - The sub-task of the issue SPARK-18507 Major performance regression in SHOW PARTITIONS on partitioned Hive tables

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #15985 [Github] Pull Request #15985 (ericl)

          Activity

            People

              ekhliang Eric Liang
              ekhliang Eric Liang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: