Details

    • Sub-task
    • Status: Closed
    • Blocker
    • Resolution: Incomplete
    • 22.01.01
    • Upcoming Branch
    • ALL COMPONENTS
    • None

    Description

      Following our discussion at https://lists.apache.org/thread/kr4v21lxx493byzgpdrzfbz3whhbm82m I ran the integration tests and found that we currently have 322 errors and 190 failures :/

      It's a blocker for releasing...

      Attachments

        1. image-2023-12-05-16-52-38-822.png
          76 kB
          Ioan Eugen Stan
        2. image-2023-12-12-18-10-16-016.png
          60 kB
          Gil Portenseigne
        3. image-2024-01-04-18-27-42-512.png
          41 kB
          Ioan Eugen Stan
        4. image-2024-01-04-18-28-27-910.png
          343 kB
          Ioan Eugen Stan

        Issue Links

          Activity

            jleroux Jacques Le Roux added a comment - That could be useful: https://docs.oracle.com/en/java/javase/17/migrate/migrating-jdk-8-later-jdk-releases.html

            Simply adding --add-opens=java.base/java.util=ALL-UNNAMED into applicationDefaultJvmArgs in build.gradle fixed the whole issues.

            jleroux Jacques Le Roux added a comment - Simply adding --add-opens=java.base/java.util=ALL-UNNAMED into applicationDefaultJvmArgs in build.gradle fixed the whole issues.

            I did not commit yet because I'm waiting for OFBIZ-12721

            jleroux Jacques Le Roux added a comment - I did not commit yet because I'm waiting for OFBIZ-12721
            ieugen Ioan Eugen Stan added a comment - - edited

            I did not have time to work on OFBIZ-12721 .
            I hope to get some time next week.

            Would love to see OFBiz with jdk17

            ieugen Ioan Eugen Stan added a comment - - edited I did not have time to work on OFBIZ-12721 . I hope to get some time next week. Would love to see OFBiz with jdk17

            Hi Eugen,

            Don't feel pressured, we have time and a workaround

            jleroux Jacques Le Roux added a comment - Hi Eugen, Don't feel pressured, we have time and a workaround

            Hi Eugen,

            Would love to see OFBiz with jdk17

            Sure I'd, I'm already running it locally with Grdl 7.6 and JDK 17. I did not cross any blocking problem so far.

            jleroux Jacques Le Roux added a comment - Hi Eugen, Would love to see OFBiz with jdk17 Sure I'd, I'm already running it locally with Grdl 7.6 and JDK 17. I did not cross any blocking problem so far.

            Just as a note has those have no effects on the tests themselves.

            When running testIntegration putting OFBiz log in a file, like testIntegration > testIntegration.log you get a number of new log information in the console. Notably a lot of those 3 warnings:

            WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.io=ALL-UNNAMED" to the JVM command line arguments to enable ObjectStream cache memory leak protection. Alternatively, you can suppress this warning by disabling ObjectStream class cache memory leak protection.
            WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.lang=ALL-UNNAMED" to the JVM command line arguments to enable ThreadLocal memory leak detection. Alternatively, you can suppress this warning by disabling ThreadLocal memory leak detection.
            WARNING: When running on Java 9 or later you need to add "--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED" to the JVM command line arguments to enable RMI Target memory leak detection. Alternatively, you can suppress this warning by disabling RMI Target memory leak detection.
            

            I don't hink we should worry about them and related in console.

            jleroux Jacques Le Roux added a comment - Just as a note has those have no effects on the tests themselves. When running testIntegration putting OFBiz log in a file, like testIntegration > testIntegration.log you get a number of new log information in the console. Notably a lot of those 3 warnings: WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.io=ALL-UNNAMED" to the JVM command line arguments to enable ObjectStream cache memory leak protection. Alternatively, you can suppress this warning by disabling ObjectStream class cache memory leak protection. WARNING: When running on Java 9 or later you need to add "--add-opens=java.base/java.lang=ALL-UNNAMED" to the JVM command line arguments to enable ThreadLocal memory leak detection. Alternatively, you can suppress this warning by disabling ThreadLocal memory leak detection. WARNING: When running on Java 9 or later you need to add "--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED" to the JVM command line arguments to enable RMI Target memory leak detection. Alternatively, you can suppress this warning by disabling RMI Target memory leak detection. I don't hink we should worry about them and related in console.

            It's logical because all OFBiz log is sent to testIntegration.log. But forgot, also because I copied only those 3, to note above that these and related warnings come from Tomcat (Catalina). That's why we don't have to worry about them.

            jleroux Jacques Le Roux added a comment - It's logical because all OFBiz log is sent to testIntegration.log. But forgot, also because I copied only those 3, to note above that these and related warnings come from Tomcat (Catalina). That's why we don't have to worry about them.

            Commit afe62e2d7cdcf753bf44e53a3426529bac869fce in ofbiz-framework's branch refs/heads/trunk from Eugen Stan
            [ https://gitbox.apache.org/repos/asf?p=ofbiz-framework.git;h=afe62e2d7c ]

            Fixed: Test package name ref for UtilObjectTests (OFBIZ-12726) (#677)

            UtilObjectTests was rererenced with a wrong package name.

            jira-bot ASF subversion and git services added a comment - Commit afe62e2d7cdcf753bf44e53a3426529bac869fce in ofbiz-framework's branch refs/heads/trunk from Eugen Stan [ https://gitbox.apache.org/repos/asf?p=ofbiz-framework.git;h=afe62e2d7c ] Fixed: Test package name ref for UtilObjectTests ( OFBIZ-12726 ) (#677) UtilObjectTests was rererenced with a wrong package name.

            Commit 565d8a37f75448eca89c345887970f7d51c08f14 in ofbiz-framework's branch refs/heads/release22.01 from Eugen Stan
            [ https://gitbox.apache.org/repos/asf?p=ofbiz-framework.git;h=565d8a37f7 ]

            Fixed: Test package name ref for UtilObjectTests (OFBIZ-12726) (#677)

            UtilObjectTests was rererenced with a wrong package name.

            jira-bot ASF subversion and git services added a comment - Commit 565d8a37f75448eca89c345887970f7d51c08f14 in ofbiz-framework's branch refs/heads/release22.01 from Eugen Stan [ https://gitbox.apache.org/repos/asf?p=ofbiz-framework.git;h=565d8a37f7 ] Fixed: Test package name ref for UtilObjectTests ( OFBIZ-12726 ) (#677) UtilObjectTests was rererenced with a wrong package name.

            Thanks Eugen,

            This was due to OFBIZ-11067

            jleroux Jacques Le Roux added a comment - Thanks Eugen, This was due to OFBIZ-11067

            When removing '--add-opens=java.base/java.util=ALL-UNNAMED' we have now 190 failures and 65 errors.

            jleroux Jacques Le Roux added a comment - When removing '--add-opens=java.base/java.util=ALL-UNNAMED' we have now 190 failures and 65 errors.

            How can i see the tests failures?

            I ran

            ```

            $ ./gradlew cleanAll loadAll testIntegration

            ```

            and opened http://localhost:63342/ofbiz/runtime/logs/test-results/html/index.html . 
            I got

            ieugen Ioan Eugen Stan added a comment - How can i see the tests failures? I ran ``` $ ./gradlew cleanAll loadAll testIntegration ``` and opened http://localhost:63342/ofbiz/runtime/logs/test-results/html/index.html .  I got

            Yes that's it, just click on the numbers

            jleroux Jacques Le Roux added a comment - Yes that's it, just click on the numbers
            ieugen Ioan Eugen Stan added a comment - - edited

            I would like to know how to run the test from IDE as it's not very clear to me.

            I have the this test failing

            with the following error, but it does not tell me where in the code the test is failing.

            A transaction error occurred reading data
                                org.xml.sax.SAXException: A transaction error occurred reading data
            org.apache.ofbiz.entity.transaction.GenericTransactionException: The 
            current transaction is marked for rollback, not beginning a new 
            transaction and aborting current operation; the rollbackOnly was caused 
            by: Failure in create operation for entity [TestingCrypto]: 
            java.lang.IllegalStateException: This object has been flagged as 
            immutable (unchangeable), probably because it came from an Entity Engine
             cache. Cannot modify an immutable entity object. Use the clone method 
            to create a mutable copy of this object.. Rolling back 
            transaction.java.lang.IllegalStateException: This object has been 
            flagged as immutable (unchangeable), probably because it came from an 
            Entity Engine cache. Cannot modify an immutable entity object. Use the 
            clone method to create a mutable copy of this object. (This object has 
            been flagged as immutable (unchangeable), probably because it came from 
            an Entity Engine cache. Cannot modify an immutable entity object. Use 
            the clone method to create a mutable copy of this object.)
                                    	at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:299)
                                    	at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:261)
                                    	at org.apache.ofbiz.testtools.EntityXmlAssertTest.run(EntityXmlAssertTest.java:80)
                                    	at org.apache.ofbiz.testtools.TestRunContainer.start(TestRunContainer.java:90)
                                    	at org.apache.ofbiz.base.container.ContainerLoader.startLoadedContainers(ContainerLoader.java:153)
                                    	at org.apache.ofbiz.base.container.ContainerLoader.load(ContainerLoader.java:77)
                                    	at org.apache.ofbiz.base.start.StartupControlPanel.loadContainers(StartupControlPanel.java:146)
                                    	at org.apache.ofbiz.base.start.StartupControlPanel.start(StartupControlPanel.java:70)
                                    	at org.apache.ofbiz.base.start.Start.main(Start.java:89)
                                    Caused by: 
            org.apache.ofbiz.entity.transaction.GenericTransactionException: The 
            current transaction is marked for rollback, not beginning a new 
            transaction and aborting current operation; the rollbackOnly was caused 
            by: Failure in create operation for entity [TestingCrypto]: 
            java.lang.IllegalStateException: This object has been flagged as 
            immutable (unchangeable), probably because it came from an Entity Engine
             cache. Cannot modify an immutable entity object. Use the clone method 
            to create a mutable copy of this object.. Rolling back 
            transaction.java.lang.IllegalStateException: This object has been 
            flagged as immutable (unchangeable), probably because it came from an 
            Entity Engine cache. Cannot modify an immutable entity object. Use the 
            clone method to create a mutable copy of this object. (This object has 
            been flagged as immutable (unchangeable), probably because it came from 
            an Entity Engine cache. Cannot modify an immutable entity object. Use 
            the clone method to create a mutable copy of this object.)
                                    	at org.apache.ofbiz.entity.transaction.TransactionUtil.begin(TransactionUtil.java:143)
                                    	at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:277)
                                    Caused by: java.lang.IllegalStateException: This
             object has been flagged as immutable (unchangeable), probably because 
            it came from an Entity Engine cache. Cannot modify an immutable entity 
            object. Use the clone method to create a mutable copy of this object.
                                    	at org.apache.ofbiz.entity.GenericEntity.assertIsMutable(GenericEntity.java:165)
                                    	at org.apache.ofbiz.entity.GenericEntity.setDelegator(GenericEntity.java:408)
                                    	at org.apache.ofbiz.entity.GenericDelegator.create(GenericDelegator.java:890)
                                    	at org.apache.ofbiz.entity.GenericDelegator.rollback(GenericDelegator.java:2715)
                                    	at org.apache.ofbiz.testtools.TestRunContainer.start(TestRunContainer.java:92)
            ieugen Ioan Eugen Stan added a comment - - edited I would like to know how to run the test from IDE as it's not very clear to me. I have the this test failing accountingtests accounting-tests-data-load Error with the following error, but it does not tell me where in the code the test is failing. A transaction error occurred reading data org.xml.sax.SAXException: A transaction error occurred reading data org.apache.ofbiz.entity.transaction.GenericTransactionException: The current transaction is marked for rollback, not beginning a new transaction and aborting current operation; the rollbackOnly was caused by: Failure in create operation for entity [TestingCrypto]: java.lang.IllegalStateException: This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object.. Rolling back transaction.java.lang.IllegalStateException: This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object. (This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object.) at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:299) at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:261) at org.apache.ofbiz.testtools.EntityXmlAssertTest.run(EntityXmlAssertTest.java:80) at org.apache.ofbiz.testtools.TestRunContainer.start(TestRunContainer.java:90) at org.apache.ofbiz.base.container.ContainerLoader.startLoadedContainers(ContainerLoader.java:153) at org.apache.ofbiz.base.container.ContainerLoader.load(ContainerLoader.java:77) at org.apache.ofbiz.base.start.StartupControlPanel.loadContainers(StartupControlPanel.java:146) at org.apache.ofbiz.base.start.StartupControlPanel.start(StartupControlPanel.java:70) at org.apache.ofbiz.base.start.Start.main(Start.java:89) Caused by: org.apache.ofbiz.entity.transaction.GenericTransactionException: The current transaction is marked for rollback, not beginning a new transaction and aborting current operation; the rollbackOnly was caused by: Failure in create operation for entity [TestingCrypto]: java.lang.IllegalStateException: This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object.. Rolling back transaction.java.lang.IllegalStateException: This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object. (This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object.) at org.apache.ofbiz.entity.transaction.TransactionUtil.begin(TransactionUtil.java:143) at org.apache.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:277) Caused by: java.lang.IllegalStateException: This object has been flagged as immutable (unchangeable), probably because it came from an Entity Engine cache. Cannot modify an immutable entity object. Use the clone method to create a mutable copy of this object. at org.apache.ofbiz.entity.GenericEntity.assertIsMutable(GenericEntity.java:165) at org.apache.ofbiz.entity.GenericEntity.setDelegator(GenericEntity.java:408) at org.apache.ofbiz.entity.GenericDelegator.create(GenericDelegator.java:890) at org.apache.ofbiz.entity.GenericDelegator.rollback(GenericDelegator.java:2715) at org.apache.ofbiz.testtools.TestRunContainer.start(TestRunContainer.java:92)

            Hi Eugen,

            If what you want is to run all integration tests in debug mode, here you go:
            https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#example-ofbiz-server-tasks
            You may prefer to run only a suite:
            https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#execute-an-integration-test-suite-in-debug-mode
            You may also prefer to run only a test case, just add --debug-jvm at the end of
            https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#execute-an-integration-test-case

            Similar READMEs exist also for stable and next branches

            HTH

            jleroux Jacques Le Roux added a comment - Hi Eugen, If what you want is to run all integration tests in debug mode, here you go: https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#example-ofbiz-server-tasks You may prefer to run only a suite: https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#execute-an-integration-test-suite-in-debug-mode You may also prefer to run only a test case, just add --debug-jvm at the end of https://nightlies.apache.org/ofbiz/trunk/readme/html5/README.html#execute-an-integration-test-case Similar READMEs exist also for stable and next branches HTH

            Thank you.

            I managed to run it in debug.

            I am more used to right click inside test and select Run/Debug option.

            Maybe we can make this happen in the future.
            It's a nicer experience.

            ieugen Ioan Eugen Stan added a comment - Thank you. I managed to run it in debug. I am more used to right click inside test and select Run/Debug option. Maybe we can make this happen in the future. It's a nicer experience.

            Hello Eugen, Jacques,

            I just join the effort, removing '--add-opens=java.base/java.util=ALL-UNNAMED', i also got the errors.

            But using :

            ./gradlew "ofbiz --test component=order "
            ./gradlew "ofbiz --test component=minilang " 

            Make the failing tests pass...  there seems to have an dependency between the tests, with one error somewhere in test or between. I will continue to investigate on Friday.

            For accounting :

            pgil Gil Portenseigne added a comment - Hello Eugen, Jacques, I just join the effort, removing '--add-opens=java.base/java.util=ALL-UNNAMED' , i also got the errors. But using : ./gradlew "ofbiz --test component=order " ./gradlew "ofbiz --test component=minilang " Make the failing tests pass...  there seems to have an dependency between the tests, with one error somewhere in test or between. I will continue to investigate on Friday. For accounting :

            Thank you pgil ,  I have put a pause on this issue as I am waiting fro a review on https://github.com/apache/ofbiz-framework/pull/678 .

            I believe building libraries from ofbiz pieces can help with clarify dependencies and after some time improve the testing support - my moving the code to more standard testing practices.

            Perhaps we could have an embedded in-memory ofbiz for testing - like we have now, but better defined in terms of dependencies.

            I believe now it's an all or nothing scenario.

            ieugen Ioan Eugen Stan added a comment - Thank you pgil ,  I have put a pause on this issue as I am waiting fro a review on https://github.com/apache/ofbiz-framework/pull/678 . I believe building libraries from ofbiz pieces can help with clarify dependencies and after some time improve the testing support - my moving the code to more standard testing practices. Perhaps we could have an embedded in-memory ofbiz for testing - like we have now, but better defined in terms of dependencies. I believe now it's an all or nothing scenario.

            So I can confirm what pgil  found.

            Running test only a single component - that normally fails - makes the tests pass.

            This is the result for running: ./gradlew cleanAll loadAll "ofbiz --test suitename=accountingtests"

            As you can see the accountingtests pass.

             

            And this is the result for running: "./gradlew cleanAll loadAll testIntegration" .

            As you can see the accountingtests fail.

             

             

            ieugen Ioan Eugen Stan added a comment - So I can confirm what pgil   found. Running test only a single component - that normally fails - makes the tests pass. This is the result for running: ./gradlew cleanAll loadAll "ofbiz --test suitename=accountingtests" As you can see the accountingtests pass.   And this is the result for running: "./gradlew cleanAll loadAll testIntegration" . As you can see the accountingtests fail.    

            I did a bit of research and found out that ./gradlew "ofbiz --test suitename=entitytests"  fails.

            It might be the thing that fails the other tests.

            I believe it's caused because xstream does not work with Java 17: https://github.com/x-stream/xstream/issues/262 .

            The recommended way to fix it is to do:
            '--add-opens=java.base/java.util=ALL-UNNAMED' // OFBIZ-12726
            which we already do.

            There does not seem to be a way around this other than replaxing xstream with something else.

            So I believe this issue can be closed and we can move forward.

            We could open a new issue, related to this where we can consider dropping xstream in place of something else.

            But I guess we can do that at a later time.

            No need to change if we have a workaround that seems ok so far.

            cc jleroux , pgil  wdyt?

             

            ```

              No converter available ---- Debugging information ---- message : No converter available type : java.util.Collections$UnmodifiableMap converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter message[1] : Unable to make field private static final long java.util.Collections$UnmodifiableMap.serialVersionUID accessible: module java.base does not "opens java.util" to unnamed module @75eeccf5 -------------------------------
             
            ```
            {{com.thoughtworks.xstream.converters.ConversionException: No converter available }}
            {{ ---- Debugging information ---- }}
            {{ message : No converter available }}
            {{ type : java.util.Collections$UnmodifiableMap }}
            {{ converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter }}
            {{ message[1] : Unable to make field
            private static final long
            java.util.Collections$UnmodifiableMap.serialVersionUID accessible:
            module java.base does not "opens java.util" to unnamed module @75eeccf5 }}
            ```
            ieugen Ioan Eugen Stan added a comment - I did a bit of research and found out that ./gradlew "ofbiz --test suitename=entitytests"  fails. It might be the thing that fails the other tests. I believe it's caused because xstream does not work with Java 17: https://github.com/x-stream/xstream/issues/262 . The recommended way to fix it is to do: '--add-opens=java.base/java.util=ALL-UNNAMED' // OFBIZ-12726 which we already do. There does not seem to be a way around this other than replaxing xstream with something else. So I believe this issue can be closed and we can move forward. We could open a new issue, related to this where we can consider dropping xstream in place of something else. But I guess we can do that at a later time. No need to change if we have a workaround that seems ok so far. cc jleroux , pgil   wdyt?   ```   No converter available ---- Debugging information ---- message : No converter available type : java.util.Collections$UnmodifiableMap converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter message [1] : Unable to make field private static final long java.util.Collections$UnmodifiableMap.serialVersionUID accessible: module java.base does not "opens java.util" to unnamed module @75eeccf5 -------------------------------   ``` {{com.thoughtworks.xstream.converters.ConversionException: No converter available }} {{ ---- Debugging information ---- }} {{ message : No converter available }} {{ type : java.util.Collections$UnmodifiableMap }} {{ converter : com.thoughtworks.xstream.converters.reflection.ReflectionConverter }} {{ message [1] : Unable to make field private static final long java.util.Collections$UnmodifiableMap.serialVersionUID accessible: module java.base does not "opens java.util" to unnamed module @75eeccf5 }} ```

            Thanks Eugen,

            I agree, Gil?

            jleroux Jacques Le Roux added a comment - Thanks Eugen, I agree, Gil?

            Thanks and nice catch, I agree we can close with creating a ticket to not forget the issue with xstream.

            pgil Gil Portenseigne added a comment - Thanks and nice catch, I agree we can close with creating a ticket to not forget the issue with xstream.

            I created OFBIZ-12877 before closing

            jleroux Jacques Le Roux added a comment - I created OFBIZ-12877 before closing

            People

              jleroux Jacques Le Roux
              jleroux Jacques Le Roux
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: