Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: Query Processor, Statistics
    • Labels:
      None

      Description

      At the first step, we gather table-level stats for non-partitioned table and partition-level stats for partitioned table. Future work could extend the table level stats to partitioned table as well.

      There are 3 major milestones in this subtask:
      1) extend the insert statement to gather table/partition level stats on-the-fly.
      2) extend metastore API to support storing and retrieving stats for a particular table/partition.
      3) add an ANALYZE TABLE [PARTITION] statement in Hive QL to gather stats for existing tables/partitions.

      The proposed stats are:

      Partition-level stats:

      • number of rows
      • total size in bytes
      • number of files
      • max, min, average row sizes
      • max, min, average file sizes

      Table-level stats in addition to partition level stats:

      • number of partitions

        Attachments

        1. HIVE-1361.2_java_only.patch
          205 kB
          Ning Zhang
        2. HIVE-1361.2.patch
          3.89 MB
          Ning Zhang
        3. HIVE-1361.3.patch
          3.88 MB
          Ning Zhang
        4. HIVE-1361.4.java_only.patch
          204 kB
          Ning Zhang
        5. HIVE-1361.4.patch
          3.88 MB
          Ning Zhang
        6. HIVE-1361.5.java_only.patch
          204 kB
          Ning Zhang
        7. HIVE-1361.5.patch
          3.88 MB
          Ning Zhang
        8. HIVE-1361.java_only.patch
          178 kB
          Ning Zhang
        9. HIVE-1361.patch
          3.67 MB
          Ning Zhang
        10. stats0.patch
          363 kB
          ahmedaly

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                nzhang Ning Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: