Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14535 add insert-only ACID tables to Hive
  3. HIVE-14990

run all tests for MM tables and fix the issues that are found

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • hive-14535
    • None
    • None

    Description

      I am running the tests with isMmTable returning true for most tables (except ACID, temporary tables, views, etc.).
      Many tests will fail because of various expected issues with such an approach; however we can find issues in MM tables from other failures.

      Expected failures
      1) All HCat tests (cannot write MM tables via the HCat writer)
      2) Almost all merge tests (alter .. concat is not supported).
      3) Tests that run dfs commands with specific paths (path changes).
      4) Truncate column (not supported).
      5) Describe formatted will have the new table fields in the output (before merging MM with ACID).
      6) Many tests w/explain extended - diff in partition "base file name" (path changes).
      7) TestTxnCommands - all the conversion tests, as they check for bucket count using file lists (path changes).
      8) HBase metastore tests cause methods are not implemented.
      9) Some load and ExIm tests that export a table and then rely on specific path for load (path changes).
      10) Bucket map join/etc. - diffs; disabled the optimization for MM tables due to how it accounts for buckets
      11) rand - different results due to different sequence of processing.
      12) many (not all i.e. not the ones with just one insert) tests that have stats output, such as file count, for obvious reasons
      13) materialized views, not handled by design - the test check erroneously makes them "mm", no easy way to tell them apart, I don't want to plumb more stuff thru just for this test

      I'm filing jiras for some test failures that are not obvious and need an investigation later

      Attachments

        1. HIVE-14990.patch
          2.65 MB
          Sergey Shelukhin
        2. HIVE-14990.01.patch
          2.63 MB
          Sergey Shelukhin
        3. HIVE-14990.02.patch
          2.63 MB
          Sergey Shelukhin
        4. HIVE-14990.03.patch
          2.64 MB
          Sergey Shelukhin
        5. HIVE-14990.04.patch
          2.73 MB
          Sergey Shelukhin
        6. HIVE-14990.04.patch
          2.73 MB
          Sergey Shelukhin
        7. HIVE-14990.05.patch
          2.73 MB
          Sergey Shelukhin
        8. HIVE-14990.05.patch
          2.73 MB
          Sergey Shelukhin
        9. HIVE-14990.06.patch
          2.77 MB
          Sergey Shelukhin
        10. HIVE-14990.07.patch
          2.91 MB
          Sergey Shelukhin
        11. HIVE-14990.06.patch
          2.91 MB
          Sergey Shelukhin
        12. HIVE-14990.08.patch
          3.07 MB
          Sergey Shelukhin
        13. HIVE-14990.09.patch
          2.99 MB
          Sergey Shelukhin
        14. HIVE-14990.10.patch
          3.25 MB
          Sergey Shelukhin
        15. HIVE-14990.10.patch
          3.25 MB
          Sergey Shelukhin
        16. HIVE-14990.10.patch
          3.25 MB
          Sergey Shelukhin
        17. HIVE-14990.12.patch
          2.71 MB
          Sergey Shelukhin
        18. HIVE-14990.13.patch
          2.71 MB
          Sergey Shelukhin
        19. HIVE-14990.14.patch
          2.71 MB
          Sergey Shelukhin
        20. HIVE-14990.15.patch
          2.72 MB
          Sergey Shelukhin
        21. HIVE-14990.16.patch
          2.72 MB
          Sergey Shelukhin
        22. HIVE-14990.17.patch
          830 kB
          Wei Zheng
        23. HIVE-14990.18.patch
          834 kB
          Wei Zheng
        24. HIVE-14990.19.patch
          837 kB
          Wei Zheng
        25. HIVE-14990.20.patch
          840 kB
          Wei Zheng
        26. HIVE-14990.21.patch
          840 kB
          Wei Zheng

        Issue Links

          Activity

            People

              wzheng Wei Zheng
              sershe Sergey Shelukhin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: