Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2484

Enable ALTER TABLE SET SERDE to work on partition level

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Allowed ALTER TABLE SET SERDE and ALTER TABLE SET SERDEPROPERTIES to accept the PARTITION qualifier to work on a specific partition.

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1004 (See https://builds.apache.org/job/Hive-trunk-h0.21/1004/)
        HIVE-2484: Enable ALTER TABLE SET SERDE to work on partition level (Xiao Li via He Yongqiang)

        heyongqiang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181197
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java
        • /hive/trunk/ql/src/test/queries/clientpositive/alter_table_serde.q
        • /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out
        Show
        hudson Hudson added a comment - Integrated in Hive-trunk-h0.21 #1004 (See https://builds.apache.org/job/Hive-trunk-h0.21/1004/ ) HIVE-2484 : Enable ALTER TABLE SET SERDE to work on partition level (Xiao Li via He Yongqiang) heyongqiang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1181197 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java /hive/trunk/ql/src/test/queries/clientpositive/alter_table_serde.q /hive/trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out
        Hide
        he yongqiang He Yongqiang added a comment -

        committed, thanks Xiao Li!

        Show
        he yongqiang He Yongqiang added a comment - committed, thanks Xiao Li!
        Hide
        he yongqiang He Yongqiang added a comment -

        +1, will commit after tests pass

        Show
        he yongqiang He Yongqiang added a comment - +1, will commit after tests pass
        Hide
        xiaol Xiao Li added a comment -

        Changed to set fields on partition level if (part != null).
        See all consolidated changes in

        HIVE-2484.patch.txt

        Show
        xiaol Xiao Li added a comment - Changed to set fields on partition level if (part != null). See all consolidated changes in HIVE-2484 .patch.txt
        Hide
        he yongqiang He Yongqiang added a comment -

        I haven't looked at partition's object to see if there is a method there. But technically, the partition's columns are copied from tables right now. So after a table's columns got changes, the old partitions can still have old ones, though they are not used today.

        Show
        he yongqiang He Yongqiang added a comment - I haven't looked at partition's object to see if there is a method there. But technically, the partition's columns are copied from tables right now. So after a table's columns got changes, the old partitions can still have old ones, though they are not used today.
        Hide
        xiaol Xiao Li added a comment -

        Thanks for the quick reply!
        Maybe I missed something but Is it right to have
        partition.setFields? Shouldn't Table only have one list of Fields
        (all partitions should share the same Fileds).

        On Wed, Oct 5, 2011 at 5:06 PM, He Yongqiang (Commented) (JIRA)

        Show
        xiaol Xiao Li added a comment - Thanks for the quick reply! Maybe I missed something but Is it right to have partition.setFields? Shouldn't Table only have one list of Fields (all partitions should share the same Fileds). On Wed, Oct 5, 2011 at 5:06 PM, He Yongqiang (Commented) (JIRA)
        Hide
        he yongqiang He Yongqiang added a comment -

        sorry, i should be clear:

        +      if (part != null) {
        +        part.getTPartition().getSd().getSerdeInfo().setSerializationLib(serdeName);
        +        if ((alterTbl.getProps() != null) && (alterTbl.getProps().size() > 0)) {
        +          part.getTPartition().getSd().getSerdeInfo().getParameters().putAll(
        +              alterTbl.getProps());
        +          }
        +        tbl.setFields(Hive.getFieldsFromDeserializer(tbl.getTableName(), tbl.
        +            getDeserializer()));
        +      } else {
        +        tbl.setSerializationLib(alterTbl.getSerdeName());
        +        if ((alterTbl.getProps() != null) && (alterTbl.getProps().size() > 0)) {
        +          tbl.getTTable().getSd().getSerdeInfo().getParameters().putAll(
        +              alterTbl.getProps());
        +          }
        +        tbl.setFields(Hive.getFieldsFromDeserializer(tbl.getTableName(), tbl.
        +            getDeserializer()));
        

        For the partition level, should not change the table object, and should change the partition object.

        Show
        he yongqiang He Yongqiang added a comment - sorry, i should be clear: + if (part != null ) { + part.getTPartition().getSd().getSerdeInfo().setSerializationLib(serdeName); + if ((alterTbl.getProps() != null ) && (alterTbl.getProps().size() > 0)) { + part.getTPartition().getSd().getSerdeInfo().getParameters().putAll( + alterTbl.getProps()); + } + tbl.setFields(Hive.getFieldsFromDeserializer(tbl.getTableName(), tbl. + getDeserializer())); + } else { + tbl.setSerializationLib(alterTbl.getSerdeName()); + if ((alterTbl.getProps() != null ) && (alterTbl.getProps().size() > 0)) { + tbl.getTTable().getSd().getSerdeInfo().getParameters().putAll( + alterTbl.getProps()); + } + tbl.setFields(Hive.getFieldsFromDeserializer(tbl.getTableName(), tbl. + getDeserializer())); For the partition level, should not change the table object, and should change the partition object.
        Hide
        xiaol Xiao Li added a comment -

        Fixed places in reviewer's comments

        Show
        xiaol Xiao Li added a comment - Fixed places in reviewer's comments
        Hide
        jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2215/#review2362
        -----------------------------------------------------------

        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        <https://reviews.apache.org/r/2215/#comment5422>

        code format here.

        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        <https://reviews.apache.org/r/2215/#comment5424>

        dup this line and move them to if (part!=null){} and else{}

        • Yongqiang

        On 2011-10-05 18:40:55, xiaol wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/2215/

        -----------------------------------------------------------

        (Updated 2011-10-05 18:40:55)

        Review request for hive and Yongqiang He.

        Summary

        -------

        Allowed ALTER TABLE SET SERDE and ALTER TABLE SET SERDEPROPERTIES to accept the PARTITION qualifier to work on a specific partition.

        This addresses bug HIVE-2484.

        https://issues.apache.org/jira/browse/HIVE-2484

        Diffs

        -----

        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1178944

        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1178944

        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1178944

        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1178944

        trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1178984

        trunk/ql/src/test/queries/clientpositive/alter_table_serde.q PRE-CREATION

        trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out PRE-CREATION

        Diff: https://reviews.apache.org/r/2215/diff

        Testing

        -------

        Thanks,

        xiaol

        Show
        jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2215/#review2362 ----------------------------------------------------------- trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java < https://reviews.apache.org/r/2215/#comment5422 > code format here. trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java < https://reviews.apache.org/r/2215/#comment5424 > dup this line and move them to if (part!=null){} and else{} Yongqiang On 2011-10-05 18:40:55, xiaol wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2215/ ----------------------------------------------------------- (Updated 2011-10-05 18:40:55) Review request for hive and Yongqiang He. Summary ------- Allowed ALTER TABLE SET SERDE and ALTER TABLE SET SERDEPROPERTIES to accept the PARTITION qualifier to work on a specific partition. This addresses bug HIVE-2484 . https://issues.apache.org/jira/browse/HIVE-2484 Diffs ----- trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1178984 trunk/ql/src/test/queries/clientpositive/alter_table_serde.q PRE-CREATION trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2215/diff Testing ------- Thanks, xiaol
        Hide
        he yongqiang He Yongqiang added a comment -

        The patch looks very good. just 2 small comments on review board.

        Show
        he yongqiang He Yongqiang added a comment - The patch looks very good. just 2 small comments on review board.
        Hide
        xiaol Xiao Li added a comment -
        Show
        xiaol Xiao Li added a comment - Review request in https://reviews.apache.org/r/2215/
        Hide
        jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2215/
        -----------------------------------------------------------

        Review request for hive and Yongqiang He.

        Summary
        -------

        Allowed ALTER TABLE SET SERDE and ALTER TABLE SET SERDEPROPERTIES to accept the PARTITION qualifier to work on a specific partition.

        This addresses bug HIVE-2484.
        https://issues.apache.org/jira/browse/HIVE-2484

        Diffs


        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1178944
        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1178944
        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1178944
        trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1178944
        trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1178984
        trunk/ql/src/test/queries/clientpositive/alter_table_serde.q PRE-CREATION
        trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out PRE-CREATION

        Diff: https://reviews.apache.org/r/2215/diff

        Testing
        -------

        Thanks,

        xiaol

        Show
        jiraposter@reviews.apache.org jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2215/ ----------------------------------------------------------- Review request for hive and Yongqiang He. Summary ------- Allowed ALTER TABLE SET SERDE and ALTER TABLE SET SERDEPROPERTIES to accept the PARTITION qualifier to work on a specific partition. This addresses bug HIVE-2484 . https://issues.apache.org/jira/browse/HIVE-2484 Diffs trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java 1178944 trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java 1178984 trunk/ql/src/test/queries/clientpositive/alter_table_serde.q PRE-CREATION trunk/ql/src/test/results/clientpositive/alter_table_serde.q.out PRE-CREATION Diff: https://reviews.apache.org/r/2215/diff Testing ------- Thanks, xiaol

          People

          • Assignee:
            xiaol Xiao Li
            Reporter:
            xiaol Xiao Li
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development