Hive
  1. Hive
  2. HIVE-6193

change partition pruning request to metastore to use list instead of set

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.13.0
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      Change partition pruning request to metastore to use list instead of set.
      Set is unwieldy w.r.t. compat, better get rid of it before API in this form was ever shipped.

      1. HIVE-6193.01.patch
        231 kB
        Sergey Shelukhin
      2. HIVE-6193.patch
        223 kB
        Sergey Shelukhin

        Activity

        Hide
        Sergey Shelukhin added a comment -

        Really trivial patch, vast majority of the changes are generated code

        Show
        Sergey Shelukhin added a comment - Really trivial patch, vast majority of the changes are generated code
        Hide
        Sergey Shelukhin added a comment -

        Ashutosh Chauhan can you take a quick look? I'd like to change this in Hive 13 (before we ship this API, so that backward compat is not a concern)

        Show
        Sergey Shelukhin added a comment - Ashutosh Chauhan can you take a quick look? I'd like to change this in Hive 13 (before we ship this API, so that backward compat is not a concern)
        Hide
        Ashutosh Chauhan added a comment -

        Why list? I think Set is a better choice here.

        Show
        Ashutosh Chauhan added a comment - Why list? I think Set is a better choice here.
        Hide
        Sergey Shelukhin added a comment -

        List is more flexible if we want to add stuff per partition and not as part of partition object (we cannot add fields specific to every request to partition object, it will become a dump).
        With set, it becomes impossible without some tricks.
        Set doesn't really do anything good for this API, it doesn't enforce logical uniqueness because it contains thrift objects with bunch of random fields that are not part of partition identity; all other partition APIs also use lists.

        Show
        Sergey Shelukhin added a comment - List is more flexible if we want to add stuff per partition and not as part of partition object (we cannot add fields specific to every request to partition object, it will become a dump). With set, it becomes impossible without some tricks. Set doesn't really do anything good for this API, it doesn't enforce logical uniqueness because it contains thrift objects with bunch of random fields that are not part of partition identity; all other partition APIs also use lists.
        Hide
        Hive QA added a comment -

        Overall: +1 all checks pass

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12622726/HIVE-6193.patch

        SUCCESS: +1 4924 tests passed

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/895/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/895/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        

        This message is automatically generated.

        ATTACHMENT ID: 12622726

        Show
        Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12622726/HIVE-6193.patch SUCCESS: +1 4924 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/895/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/895/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12622726
        Hide
        Ashutosh Chauhan added a comment -

        +1

        Show
        Ashutosh Chauhan added a comment - +1
        Hide
        Sergey Shelukhin added a comment -

        Rebase patch to resolve conflicts (in thrift generation); also removed some temporary TODO-s left from some other patch

        Show
        Sergey Shelukhin added a comment - Rebase patch to resolve conflicts (in thrift generation); also removed some temporary TODO-s left from some other patch
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk. Thanks, Sergey!

        Show
        Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Sergey!

          People

          • Assignee:
            Sergey Shelukhin
            Reporter:
            Sergey Shelukhin
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development