Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3125

TestDBLoadDump.testImportInvalidDataLeavesTablesEmpty fails

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 5.0.0b1
    • tests
    • None

    Description

      Running mvn test -Dtest=TestDBLoadDump#testImportInvalidDataLeavesTablesEmpty results in a test failure:

      [ERROR]   TestDBLoadDump.testImportInvalidDataLeavesTablesEmpty:136->tryImportAndCheckPrematureExit:271 import should have been ended prematurely
      

      TestDBLoadDump.testImportInvalidDataLeavesTablesEmpty tests that processing an invalid dump file results in transactions are rolled back and tables are left unpopulated (empty).

      tools/src/test/resources/dumpData/invalid/ooziedb_ac.json contains an executionPath that consists of more than 1024 characters (2000).

      OozieDBImportCLI.importOneInputFileToOneEntityTable deserialize the WorkflowActionBean object from the JSON using final E newEntity = gson.fromJson(line, entityClass); . However, persisting this object should fail even because invariants (see @Column(name = "execution_path", length = 1024) in WorkflowActionBean) do not hold.

      In the setup method of TestDBLoadDump, setSqlStricEnforce() is used to make sure strict size is enforced:

          private void setSqlStrictEnforce(final EntityManager entityManager) {
              final String sqlStrictEnforce = "SET PROPERTY \"sql.enforce_strict_size\" TRUE";
      
              final EntityTransaction tx = entityManager.getTransaction();
              tx.begin();
      
              entityManager.createNativeQuery(sqlStrictEnforce).executeUpdate();
      
              tx.commit();
          }
      

      However it does not seem to take effect. HSQL db's documentation says:

      Management of properties has changed since version 1.8. The old SET PROPERTY statement does not change a property and is ignored. The statement is retained to simplify application upgrades.

      Running mvn dependency:tree, it turns out Oozie core uses 1.8.0.10

      [INFO] +- hsqldb:hsqldb:jar:1.8.0.10:compile
      

      It is, however, strange that tests pass if running all the test cases in TestDBLoadDump with mvn test -Dmaven.surefire.debug -Dtest=TestDBLoadDump. In other words, tests are not independent of execution order.

      Looking at a successfull test execution in surefire-reports (TEST-org.apache.oozie.tools.TestDBLoadDump.xml) :

        <testcase name="testImportTablesOverflowBatchSize" classname="org.apache.oozie.tools.TestDBLoadDump" time="25.362"/>
        <testcase name="testImportToNonExistingTablesSucceedsOnHsqldb" classname="org.apache.oozie.tools.TestDBLoadDump" time="0.559"/>
        <testcase name="testImportInvalidDataLeavesTablesEmpty" classname="org.apache.oozie.tools.TestDBLoadDump" time="0.708"/>
        <testcase name="testImportToNonEmptyTablesCausesPrematureExit" classname="org.apache.oozie.tools.TestDBLoadDump" time="1.066"/>
        <testcase name="testImportedDBIsExportedCorrectly" classname="org.apache.oozie.tools.TestDBLoadDump" time="1.135"/>
      

      We shall make sure individual tests pass. As a first step, TestDBLoadDump.testImportInvalidDataLeavesTablesEmpty is to be fixed.

      Attachments

        1. OOZIE-3125-001.patch
          0.6 kB
          Attila Sasvári
        2. OOZIE-3125-002.patch
          3 kB
          Attila Sasvári
        3. OOZIE-3125-amend-1.patch
          3 kB
          Attila Sasvári
        4. OOZIE-3125-amend-2.patch
          6 kB
          Attila Sasvári

        Issue Links

          Activity

            People

              asasvari Attila Sasvári
              asasvari Attila Sasvári
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: