Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19883

QTestUtil: initDataset() can be affected by the settings of the previous test

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Testing Infrastructure
    • Labels:
      None

      Description

      Tried creating a test that set metastore.create.as.acid/hive.create.as.insert.only, and I found that the built-in table default.src was being created as an insert-only transactional table, which will cause errors in other tests that do not set the TxnManager to one that supports transactional tables.

      It appears that initDataset() uses the old CliDriver that was used for the previous test, which has any settings used during that test:

      java.lang.Exception: Creating src
              at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4926) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:428) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2659) [hive-exec-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2311) [hive-exec-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1982) [hive-exec-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1683) [hive-exec-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1677) [hive-exec-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:218) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) [hive-cli-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188) [hive-cli-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402) [hive-cli-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335) [hive-cli-4.0.0-SNAPSHOT.jar:?]
              at org.apache.hadoop.hive.ql.QTestUtil.initDataset(QTestUtil.java:1277) [classes/:?]
              at org.apache.hadoop.hive.ql.QTestUtil.initDataSetForTest(QTestUtil.java:1259) [classes/:?]
              at org.apache.hadoop.hive.ql.QTestUtil.cliInit(QTestUtil.java:1328) [classes/:?]
              at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:176) [classes/:?]
              at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104) [classes/:?]
              at org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:59) [test-classes/:?]
      

      A new CliDriver is created for the new test, but only after we've created the dataset tables for the next test (see the line numbers for QTestUtil.cliInit() in both stack traces).

      CliSessionState(SessionState).getConf() line: 317	
      CliDriver.<init>() line: 110	
      QTestUtil.cliInit(File, boolean) line: 1360	
      CoreCliDriver.runTest(String, String, String) line: 176	
      CoreCliDriver(CliAdapter).runTest(String, File) line: 104	
      TestMiniLlapLocalCliDriver.testCliDriver() line: 59	
      

      I think fix is to move the creation of the new CliDriver higher up in QTestUtil.cliInit(), before we call initDataset().

        Attachments

        1. HIVE-19883.1.patch
          2 kB
          Jason Dere
        2. HIVE-19883.2.patch
          5 kB
          Jason Dere

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment