Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21954

QTest: support for running qtests on various metastore DBs

    XMLWordPrintableJSON

    Details

      Description

      In HIVE-21940, a postgres metastore related issue has been fixed, and a local reproduction has been provided.

      export QTEST_LEAVE_FILES=true
      docker kill metastore-test-postgres-install
      docker rm metastore-test-postgres-install
      cd standalone-metastore
      mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch -Dmetastore.itest.no.stop.container=true
      cd ..
      mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
      

      The problem with this solution is that data/conf/hive-site.xml has to be edited manually. My proposal is to introduce a property (-Dmetastore.db=postgres), which can take care of the parameters on the fly. 2 supported solutions could be:
      1. simple parameters: -Dmetastore.db=postgres
      In this case, tests depend on settings from ITestPostgres class (password, db, etc.)
      2. verbose but flexible parameters: see hive-site.xml HIVE-21940's repro patch

      In the long term, we should consider running this kind of tests in precommit phase, so maybe -Dmetastore.db=postgres could start metastore db automatically. Also we should consider running some qtests on various metastores. I would not pick randomly, but choose some "metastore-heavy" ones instead.

        Attachments

        1. HIVE-21954.10.patch
          93 kB
          László Bodor
        2. HIVE-21954.10.patch
          93 kB
          László Bodor
        3. HIVE-21954.09.patch
          94 kB
          László Bodor
        4. HIVE-21954.08.patch
          93 kB
          László Bodor
        5. HIVE-21954.07.patch
          92 kB
          László Bodor
        6. HIVE-21954.07.patch
          92 kB
          László Bodor
        7. HIVE-21954.05.patch
          88 kB
          László Bodor
        8. HIVE-21954.04.patch
          88 kB
          László Bodor
        9. HIVE-21954.03.patch
          87 kB
          László Bodor
        10. HIVE-21954.03.patch
          87 kB
          László Bodor
        11. HIVE-21954.03.patch
          87 kB
          László Bodor
        12. HIVE-21954.02.patch
          85 kB
          László Bodor
        13. HIVE-21954.01.patch
          79 kB
          László Bodor

          Activity

            People

            • Assignee:
              abstractdog László Bodor
              Reporter:
              abstractdog László Bodor
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: