Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HIVE-2215 adds the ability to add events in metastore. These events needs to be purged as they have limited life.

      1. hive-2225_1.patch
        10 kB
        Ashutosh Chauhan
      2. hive-2225.patch
        8 kB
        Ashutosh Chauhan

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #801 (See https://builds.apache.org/job/Hive-trunk-h0.21/801/)
        HIVE-2225. Purge expired metastore events (Ashutosh Chauhan via cws)

        cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1141430
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java
        • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
        • /hive/trunk/conf/hive-default.xml
        • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
        • /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
        • /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #801 (See https://builds.apache.org/job/Hive-trunk-h0.21/801/ ) HIVE-2225 . Purge expired metastore events (Ashutosh Chauhan via cws) cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1141430 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java /hive/trunk/conf/hive-default.xml /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java
        Hide
        Carl Steinbach added a comment -

        Committed to trunk. Thanks Ashutosh!

        Show
        Carl Steinbach added a comment - Committed to trunk. Thanks Ashutosh!
        Hide
        Ashutosh Chauhan added a comment -

        Sorry.. RB-Jira sync. Had the patch on RB. Reattaching it here.

        Show
        Ashutosh Chauhan added a comment - Sorry.. RB-Jira sync. Had the patch on RB. Reattaching it here.
        Hide
        Carl Steinbach added a comment -

        @Ashutosh: can you please attach the most recent version of the patch? Thanks.

        Show
        Carl Steinbach added a comment - @Ashutosh: can you please attach the most recent version of the patch? Thanks.
        Hide
        Carl Steinbach added a comment -

        +1. Will commit if tests pass.

        Show
        Carl Steinbach added a comment - +1. Will commit if tests pass.
        Hide
        Carl Steinbach added a comment -

        Looks good to me.

        Show
        Carl Steinbach added a comment - Looks good to me.
        Hide
        Carl Steinbach added a comment -

        Looking at it now.

        Show
        Carl Steinbach added a comment - Looking at it now.
        Hide
        John Sichi added a comment -

        Carl, any comments on this one?

        Show
        John Sichi added a comment - Carl, any comments on this one?
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-06-22 23:12:49, John Sichi wrote:

        > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 222

        > <https://reviews.apache.org/r/940/diff/1/?file=21411#file21411line222>

        >

        > Existing property names are "hive.metastore..." instead of just "metastore..."

        done

        • Ashutosh

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review890
        -----------------------------------------------------------

        On 2011-06-23 02:55:08, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-23 02:55:08)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719

        trunk/conf/hive-default.xml 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-06-22 23:12:49, John Sichi wrote: > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 222 > < https://reviews.apache.org/r/940/diff/1/?file=21411#file21411line222 > > > Existing property names are "hive.metastore..." instead of just "metastore..." done Ashutosh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review890 ----------------------------------------------------------- On 2011-06-23 02:55:08, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-23 02:55:08) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719 trunk/conf/hive-default.xml 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-06-22 23:07:46, John Sichi wrote:

        > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 221

        > <https://reviews.apache.org/r/940/diff/1/?file=21411#file21411line221>

        >

        > If you agree about making this disabled by default, we could use a special value such as 0 for the frequency to indicate disabled.

        >

        Done. Timer is now created only if this property has non-zero value.

        • Ashutosh

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review889
        -----------------------------------------------------------

        On 2011-06-23 02:55:08, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-23 02:55:08)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719

        trunk/conf/hive-default.xml 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-06-22 23:07:46, John Sichi wrote: > trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 221 > < https://reviews.apache.org/r/940/diff/1/?file=21411#file21411line221 > > > If you agree about making this disabled by default, we could use a special value such as 0 for the frequency to indicate disabled. > Done. Timer is now created only if this property has non-zero value. Ashutosh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review889 ----------------------------------------------------------- On 2011-06-23 02:55:08, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-23 02:55:08) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719 trunk/conf/hive-default.xml 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-06-22 23:18:43, John Sichi wrote:

        > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 259

        > <https://reviews.apache.org/r/940/diff/1/?file=21412#file21412line259>

        >

        > Why is this using a Thread instead of a Timer?

        Agreed timer is better suited here then Thread. Changed to timer.

        On 2011-06-22 23:18:43, John Sichi wrote:

        > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java, line 33

        > <https://reviews.apache.org/r/940/diff/1/?file=21415#file21415line33>

        >

        > 6 hrs is actually configurable, right?

        Yup, it is.

        • Ashutosh

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review891
        -----------------------------------------------------------

        On 2011-06-23 02:55:08, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-23 02:55:08)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719

        trunk/conf/hive-default.xml 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-06-22 23:18:43, John Sichi wrote: > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 259 > < https://reviews.apache.org/r/940/diff/1/?file=21412#file21412line259 > > > Why is this using a Thread instead of a Timer? Agreed timer is better suited here then Thread. Changed to timer. On 2011-06-22 23:18:43, John Sichi wrote: > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java, line 33 > < https://reviews.apache.org/r/940/diff/1/?file=21415#file21415line33 > > > 6 hrs is actually configurable, right? Yup, it is. Ashutosh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review891 ----------------------------------------------------------- On 2011-06-23 02:55:08, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-23 02:55:08) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719 trunk/conf/hive-default.xml 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/
        -----------------------------------------------------------

        (Updated 2011-06-23 02:55:08.540561)

        Review request for hive, Carl Steinbach and John Sichi.

        Changes
        -------

        Updated the patch per John's comments.

        Summary
        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.
        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs (updated)


        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719
        trunk/conf/hive-default.xml 1138719
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION
        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719

        Diff: https://reviews.apache.org/r/940/diff

        Testing
        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-23 02:55:08.540561) Review request for hive, Carl Steinbach and John Sichi. Changes ------- Updated the patch per John's comments. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs (updated) trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138719 trunk/conf/hive-default.xml 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138719 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerTask.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138719 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2011-06-22 23:07:05, John Sichi wrote:

        > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java, line 1

        > <https://reviews.apache.org/r/940/diff/1/?file=21415#file21415line1>

        >

        > New files need Apache headers

        Added.

        • Ashutosh

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review888
        -----------------------------------------------------------

        On 2011-06-21 17:34:28, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-21 17:34:28)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - On 2011-06-22 23:07:05, John Sichi wrote: > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java, line 1 > < https://reviews.apache.org/r/940/diff/1/?file=21415#file21415line1 > > > New files need Apache headers Added. Ashutosh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review888 ----------------------------------------------------------- On 2011-06-21 17:34:28, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-21 17:34:28) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review891
        -----------------------------------------------------------

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
        <https://reviews.apache.org/r/940/#comment1932>

        Why is this using a Thread instead of a Timer?

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java
        <https://reviews.apache.org/r/940/#comment1931>

        6 hrs is actually configurable, right?

        • John

        On 2011-06-21 17:34:28, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-21 17:34:28)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review891 ----------------------------------------------------------- trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java < https://reviews.apache.org/r/940/#comment1932 > Why is this using a Thread instead of a Timer? trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java < https://reviews.apache.org/r/940/#comment1931 > 6 hrs is actually configurable, right? John On 2011-06-21 17:34:28, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-21 17:34:28) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review890
        -----------------------------------------------------------

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        <https://reviews.apache.org/r/940/#comment1930>

        Existing property names are "hive.metastore..." instead of just "metastore..."

        • John

        On 2011-06-21 17:34:28, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-21 17:34:28)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review890 ----------------------------------------------------------- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java < https://reviews.apache.org/r/940/#comment1930 > Existing property names are "hive.metastore..." instead of just "metastore..." John On 2011-06-21 17:34:28, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-21 17:34:28) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review888
        -----------------------------------------------------------

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java
        <https://reviews.apache.org/r/940/#comment1928>

        New files need Apache headers

        • John

        On 2011-06-21 17:34:28, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-21 17:34:28)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review888 ----------------------------------------------------------- trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java < https://reviews.apache.org/r/940/#comment1928 > New files need Apache headers John On 2011-06-21 17:34:28, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-21 17:34:28) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/#review889
        -----------------------------------------------------------

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        <https://reviews.apache.org/r/940/#comment1929>

        If you agree about making this disabled by default, we could use a special value such as 0 for the frequency to indicate disabled.

        • John

        On 2011-06-21 17:34:28, Ashutosh Chauhan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/940/

        -----------------------------------------------------------

        (Updated 2011-06-21 17:34:28)

        Review request for hive, Carl Steinbach and John Sichi.

        Summary

        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.

        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099

        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION

        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing

        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/#review889 ----------------------------------------------------------- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java < https://reviews.apache.org/r/940/#comment1929 > If you agree about making this disabled by default, we could use a special value such as 0 for the frequency to indicate disabled. John On 2011-06-21 17:34:28, Ashutosh Chauhan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- (Updated 2011-06-21 17:34:28) Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        John Sichi added a comment -

        Let's make it possible to disable the thread altogether, and make that the default for out-of-the-box Hive.

        Show
        John Sichi added a comment - Let's make it possible to disable the thread altogether, and make that the default for out-of-the-box Hive.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/940/
        -----------------------------------------------------------

        Review request for hive, Carl Steinbach and John Sichi.

        Summary
        -------

        This addresses HIVE-2225

        This addresses bug HIVE-2225.
        https://issues.apache.org/jira/browse/HIVE-2225

        Diffs


        trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099
        trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION
        trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099

        Diff: https://reviews.apache.org/r/940/diff

        Testing
        -------

        updated a test case which exercises this code path.

        Thanks,

        Ashutosh

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/940/ ----------------------------------------------------------- Review request for hive, Carl Steinbach and John Sichi. Summary ------- This addresses HIVE-2225 This addresses bug HIVE-2225 . https://issues.apache.org/jira/browse/HIVE-2225 Diffs trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1138099 trunk/metastore/src/java/org/apache/hadoop/hive/metastore/events/EventCleanerThread.java PRE-CREATION trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestMarkPartition.java 1138099 Diff: https://reviews.apache.org/r/940/diff Testing ------- updated a test case which exercises this code path. Thanks, Ashutosh
        Hide
        Ashutosh Chauhan added a comment -

        This patch is on top of HIVE-2215. It wakes up after every 6 hours (configurable) and deletes all the events which are older then 7 days(configurable).

        Show
        Ashutosh Chauhan added a comment - This patch is on top of HIVE-2215 . It wakes up after every 6 hours (configurable) and deletes all the events which are older then 7 days(configurable).

          People

          • Assignee:
            Ashutosh Chauhan
            Reporter:
            Ashutosh Chauhan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development