Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Views
    • Labels:
      None

      Description

      Materialized views are useful as ways to store either alternate versions of data (e.g. same data, different sort order) or derivatives of data sets (e.g. commonly used aggregates). It is useful to store these as materialized views rather than as tables because it can give the optimizer the ability to understand how data sets are related.

        Issue Links

          Activity

          Hide
          alangates Alan Gates added a comment -

          This patch is a start at implementing simple views. It doesn't have enough testing yet (e.g. there's no negative testing). And I know it fails in the partitioned case. I suspect things like security and locking don't work properly yet either. But I'm posting it as a starting point.

          In this initial patch I'm just handling simple materialized views with manual rebuilds. In later JIRAs we can add features such as allowing the optimizer to rewrite queries to use materialized views rather than tables named in the queries, giving the optimizer the ability to determine when a materialized view is stale, etc.

          Also, I didn't rebase this patch against trunk after the migration from svn->git so it may not apply cleanly.

          Show
          alangates Alan Gates added a comment - This patch is a start at implementing simple views. It doesn't have enough testing yet (e.g. there's no negative testing). And I know it fails in the partitioned case. I suspect things like security and locking don't work properly yet either. But I'm posting it as a starting point. In this initial patch I'm just handling simple materialized views with manual rebuilds. In later JIRAs we can add features such as allowing the optimizer to rewrite queries to use materialized views rather than tables named in the queries, giving the optimizer the ability to determine when a materialized view is stale, etc. Also, I didn't rebase this patch against trunk after the migration from svn->git so it may not apply cleanly.
          Hide
          alangates Alan Gates added a comment -

          New version of the patch with many more tests and security added.

          Show
          alangates Alan Gates added a comment - New version of the patch with many more tests and security added.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12732895/HIVE-10459.2.patch

          ERROR: -1 due to 2 failed/errored test(s), 8940 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_static
          org.apache.hadoop.hive.ql.TestErrorMsg.testUniqueErrorCode
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3894/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3894/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3894/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12732895 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12732895/HIVE-10459.2.patch ERROR: -1 due to 2 failed/errored test(s), 8940 tests executed Failed tests: org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_static org.apache.hadoop.hive.ql.TestErrorMsg.testUniqueErrorCode Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3894/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3894/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3894/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed This message is automatically generated. ATTACHMENT ID: 12732895 - PreCommit-HIVE-TRUNK-Build
          Hide
          vgumashta Vaibhav Gumashta added a comment - - edited

          Alan Gates Can you create an RB link / pull request as well? Thanks.

          Show
          vgumashta Vaibhav Gumashta added a comment - - edited Alan Gates Can you create an RB link / pull request as well? Thanks.
          Hide
          jcamachorodriguez Jesus Camacho Rodriguez added a comment -

          Alan Gates, I will move the initial work to HIVE-14249; we can follow-up there with anything related to the patch. As there are a few people watching this issue, I will use this as the umbrella JIRA for all the work that we will do around materialized views implementation.

          Show
          jcamachorodriguez Jesus Camacho Rodriguez added a comment - Alan Gates , I will move the initial work to HIVE-14249 ; we can follow-up there with anything related to the patch. As there are a few people watching this issue, I will use this as the umbrella JIRA for all the work that we will do around materialized views implementation.

            People

            • Assignee:
              jcamachorodriguez Jesus Camacho Rodriguez
              Reporter:
              alangates Alan Gates
            • Votes:
              3 Vote for this issue
              Watchers:
              27 Start watching this issue

              Dates

              • Created:
                Updated:

                Development