Hive
  1. Hive
  2. HIVE-2471

Add timestamp column to the partition stats table.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Occasionally, when entries are added to the partition stats table the program is halted before it can delete those entries, by an exception, keyboard interrupt, etc. These build up to the point where the table gets very large, and it hurts the performance of the update statement which is often called. In order to fix this, I am adding a column to the table which is auto-populated with the current timestamp. This will allow us to create scripts that go through periodically and clean out old entries from the table.

        Activity

        Ashutosh Chauhan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Ashutosh Chauhan made changes -
        Fix Version/s 0.9.0 [ 12317742 ]
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Phabricator made changes -
        Attachment HIVE-2471.D2367.3.patch [ 12518736 ]
        Phabricator made changes -
        Attachment HIVE-2471.D2367.2.patch [ 12518722 ]
        Kevin Wilfong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Kevin Wilfong made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Kevin Wilfong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Kevin Wilfong made changes -
        Description Occasionally, when entries are added to the partition stats table the program is halted before it can delete those entries, by an exception, keyboard interrupt, etc. These build up to the point where the table gets very large, and it hurts the performance of the update statement which is often called. In order to fix this, I am adding a column to the table which is auto-populated with the current timestamp. I am also adding an index on this column. This will allow us to create scripts that go through periodically and clean out old entries from the table. Occasionally, when entries are added to the partition stats table the program is halted before it can delete those entries, by an exception, keyboard interrupt, etc. These build up to the point where the table gets very large, and it hurts the performance of the update statement which is often called. In order to fix this, I am adding a column to the table which is auto-populated with the current timestamp. This will allow us to create scripts that go through periodically and clean out old entries from the table.
        Kevin Wilfong made changes -
        Summary Add timestamp column with index to the partition stats table. Add timestamp column to the partition stats table.
        Description Occasionally, when entries are added to the partition stats table the program is halted before it can delete those entries, by an exception, keyboard interrupt, etc. These build up to the point where the table gets very large, and it hurts the performance of the update statement which is often called. In order to fix this, I am adding a column to the table which is auto-populated with the current timestamp. I am also adding an index on this column. This will allow us to create scripts that go through periodically and clean out old entries from the table. The index will help to keep the runtime of these scripts short, and hence reduce the amount of time they need to lock the table/indexes for. Occasionally, when entries are added to the partition stats table the program is halted before it can delete those entries, by an exception, keyboard interrupt, etc. These build up to the point where the table gets very large, and it hurts the performance of the update statement which is often called. In order to fix this, I am adding a column to the table which is auto-populated with the current timestamp. I am also adding an index on this column. This will allow us to create scripts that go through periodically and clean out old entries from the table.
        Phabricator made changes -
        Attachment HIVE-2471.D2367.1.patch [ 12518716 ]
        Kevin Wilfong made changes -
        Field Original Value New Value
        Attachment HIVE-2471.1.patch.txt [ 12496816 ]
        Kevin Wilfong created issue -

          People

          • Assignee:
            Kevin Wilfong
            Reporter:
            Kevin Wilfong
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development