Harmony
  1. Harmony
  2. HARMONY-2772

[drlvm][gc] test stress.Mix intermittently failed on the em64t platform

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: DRLVM
    • Labels:
      None
    • Environment:
      Linux / x86_64

      Description

      Test stress.Mix intermittently failed on the em64t platform with "Segmentation fault".

      Execution log:
      vivanov1@nstdrlel18:/export/users/viv/trunk/cc/projects/drlvm/trunk/build/lnx_em64t_gcc_debug/semis/vm/_smoke.tests/classes> ../../../../deploy/jdk/jre/bin/java -cp . stress.Mix
      oC............................................................ccd+saeuluccd+saleuccd+saleuccd+saleucd+sacleuccd+saleucd+sc
      PASSED
      vivanov1@nstdrlel18:/export/users/viv/trunk/cc/projects/drlvm/trunk/build/lnx_em64t_gcc_debug/semis/vm/_smoke.tests/classes> ../../../../deploy/jdk/jre/bin/java -cp . stress.Mix
      Co............................................................uccd+saleuccd+saleuccd+saleuccd+saleuccd+saleuccd+salSegmentation fault
      vivanov1@nstdrlel18:/export/users/viv/trunk/cc/projects/drlvm/trunk/build/lnx_em64t_gcc_debug/semis/vm/_smoke.tests/classes>

      1. HARMONY-2772_GC_assert.patch
        3 kB
        Pavel Afremov
      2. HARMONY-2772_GC_assert.patch
        6 kB
        Pavel Afremov

        Issue Links

          Activity

          Hide
          Xiao-Feng Li added a comment -

          Closed since it's not reproducible.

          Show
          Xiao-Feng Li added a comment - Closed since it's not reproducible.
          Hide
          Pavel Afremov added a comment -

          I can't reproduce the bug too. I ran it 1000 times and it passed always.

          Show
          Pavel Afremov added a comment - I can't reproduce the bug too. I ran it 1000 times and it passed always.
          Hide
          Ilya Berezhniuk added a comment -

          I've performed 1000 runs of stress.Mix in cycle, all 1000 runs are successful (PASSED).

          The only problem I saw is exception message:
          --------------------------------
          Co............................................................uc+cdsuacecd+salleuccds+alecud+saleuccc+sdaleuccd+saleu
          allocate6 terminated by java.lang.NullPointerException
          cjava.lang.NullPointerException
          at stress.Mix.allocate(Mix.java:281)
          at stress.Mix.run(Mix.java:74)
          cd+sd+ad+aleuccd+saleuccd+saleuccd+saleuccd+saleuccd+s
          PASSED
          --------------------------------
          But the test is passed anyway.

          I've also added assertions to GC_GEN to check if GC is called in disabled mode (with filtering GC threads, of course).
          These assertions did not occur for me on 1000 runs.

          Show
          Ilya Berezhniuk added a comment - I've performed 1000 runs of stress.Mix in cycle, all 1000 runs are successful (PASSED). The only problem I saw is exception message: -------------------------------- Co............................................................uc+cdsuacecd+salleuccds+alecud+saleuccc+sdaleuccd+saleu allocate6 terminated by java.lang.NullPointerException cjava.lang.NullPointerException at stress.Mix.allocate(Mix.java:281) at stress.Mix.run(Mix.java:74) cd+sd+ad+aleuccd+saleuccd+saleuccd+saleuccd+saleuccd+s PASSED -------------------------------- But the test is passed anyway. I've also added assertions to GC_GEN to check if GC is called in disabled mode (with filtering GC threads, of course). These assertions did not occur for me on 1000 runs.
          Hide
          Xiao-Feng Li added a comment -

          It's more like a threading bug, because
          1. the failure happens in both gc_cc and gc_gen;
          2. the processing of finref depends on the thread scheduling.

          Show
          Xiao-Feng Li added a comment - It's more like a threading bug, because 1. the failure happens in both gc_cc and gc_gen; 2. the processing of finref depends on the thread scheduling.
          Hide
          Gregory Shimansky added a comment -

          Reassigned to gc based on Pavel's comment.

          Show
          Gregory Shimansky added a comment - Reassigned to gc based on Pavel's comment.
          Hide
          Pavel Afremov added a comment -

          Attached patch leads to assert in GC like in bug description, but it's not intermittent.
          Does anybody know can object have null value of vt or not?

          Show
          Pavel Afremov added a comment - Attached patch leads to assert in GC like in bug description, but it's not intermittent. Does anybody know can object have null value of vt or not?
          Hide
          Pavel Afremov added a comment -

          I add assertion in GC to find where object with null vt added to the pools.

          Show
          Pavel Afremov added a comment - I add assertion in GC to find where object with null vt added to the pools.
          Hide
          Pavel Afremov added a comment -

          I've reproduced this bug with new gc_gen on the following stack.:
          ...
          #7 0x00002b4825189246 in __assert_fail () from /lib64/libc.so.6
          #8 0x00002aaaad5016e4 in obj_is_marked_in_vt (obj=0x2aaab3c28d40)
          at .../vm/gc_gen/src/common/gc_for_class.h:200
          #9 0x00002aaaad4fe834 in nondestructively_fix_finref_pool (gc=0x56c0a0, pool=0x587aa0, pointer_addr_in_pool=1)
          at .../vm/gc_gen/src/finalizer_weakref/finalizer_weakref.cpp:730
          #10 0x00002aaaad4fe973 in gc_update_finref_repointed_refs (gc=0x56c0a0)
          at .../vm/gc_gen/src/finalizer_weakref/finalizer_weakref.cpp:746
          #11 0x00002aaaad51f631 in gc_fix_rootset (collector=0x58a1c0)
          at .../vm/gc_gen/src/common/gc_metadata.cpp:225
          #12 0x00002aaaad539aaf in slide_compact_mspace (collector=0x58a1c0)
          at .../vm/gc_gen/src/mark_compact/mspace_slide_compact.cpp:507
          #13 0x00002aaaad4fa90f in collector_thread_func (arg=0x58a1c0)
          at .../vm/gc_gen/src/thread/collector.cpp:186
          #14 0x00002b4824c41300 in hythread_wrapper_start_proc (arg=0x58a650)
          at .../vm/thread/src/thread_native_basic.c:725
          #15 0x00002b48258fb143 in start_thread () from /lib64/libpthread.so.0

          Assertion happened because null vt_raw value in the object.
          Despite the fact that bug reproduced on both gc_cc and gc_gen, I suppose that it is bug in gc, because repset and finalize and refs pools filled up inside GC only.

          Show
          Pavel Afremov added a comment - I've reproduced this bug with new gc_gen on the following stack.: ... #7 0x00002b4825189246 in __assert_fail () from /lib64/libc.so.6 #8 0x00002aaaad5016e4 in obj_is_marked_in_vt (obj=0x2aaab3c28d40) at .../vm/gc_gen/src/common/gc_for_class.h:200 #9 0x00002aaaad4fe834 in nondestructively_fix_finref_pool (gc=0x56c0a0, pool=0x587aa0, pointer_addr_in_pool=1) at .../vm/gc_gen/src/finalizer_weakref/finalizer_weakref.cpp:730 #10 0x00002aaaad4fe973 in gc_update_finref_repointed_refs (gc=0x56c0a0) at .../vm/gc_gen/src/finalizer_weakref/finalizer_weakref.cpp:746 #11 0x00002aaaad51f631 in gc_fix_rootset (collector=0x58a1c0) at .../vm/gc_gen/src/common/gc_metadata.cpp:225 #12 0x00002aaaad539aaf in slide_compact_mspace (collector=0x58a1c0) at .../vm/gc_gen/src/mark_compact/mspace_slide_compact.cpp:507 #13 0x00002aaaad4fa90f in collector_thread_func (arg=0x58a1c0) at .../vm/gc_gen/src/thread/collector.cpp:186 #14 0x00002b4824c41300 in hythread_wrapper_start_proc (arg=0x58a650) at .../vm/thread/src/thread_native_basic.c:725 #15 0x00002b48258fb143 in start_thread () from /lib64/libpthread.so.0 Assertion happened because null vt_raw value in the object. Despite the fact that bug reproduced on both gc_cc and gc_gen, I suppose that it is bug in gc, because repset and finalize and refs pools filled up inside GC only.
          Hide
          Gregory Shimansky added a comment -

          Quite possibly the crash happens for the same reason as in HARMONY-3338.

          Show
          Gregory Shimansky added a comment - Quite possibly the crash happens for the same reason as in HARMONY-3338 .
          Hide
          Elena Semukhina added a comment -

          I also saw Segmentation fault when running stress.Mix in interpreter mode:

          Co............................................................uccdal+sued+caeluccdsc+saleuccd+saleuSegmentation fault

          It happened after 518 successful runs.

          Show
          Elena Semukhina added a comment - I also saw Segmentation fault when running stress.Mix in interpreter mode: Co............................................................uccdal+sued+caeluccdsc+saleuccd+saleuSegmentation fault It happened after 518 successful runs.
          Hide
          Elena Semukhina added a comment -

          I see intermittent failures like the following:

          Co............................................................ucc+saeluccdd+java: /nfs/ins/proj/drl/coreapi/esemukhi/em64/drlvm/trunk/vm/vmcore/include/object_layout.h:107: VTable* ManagedObject::vt(): Assertion `vt_offset' failed.
          sFatal error
          It looks like Comment#1 but it does not print seg fault.

          Another failure I see looks like the test suddenly finishes too early and does not print PASSED so is reported as FAILED:

          oC............................................................uccd+saeluccd+saeulccd

          Show
          Elena Semukhina added a comment - I see intermittent failures like the following: Co............................................................ucc+saeluccdd+java: /nfs/ins/proj/drl/coreapi/esemukhi/em64/drlvm/trunk/vm/vmcore/include/object_layout.h:107: VTable* ManagedObject::vt(): Assertion `vt_offset' failed. sFatal error It looks like Comment#1 but it does not print seg fault. Another failure I see looks like the test suddenly finishes too early and does not print PASSED so is reported as FAILED: oC............................................................uccd+saeluccd+saeulccd
          Hide
          Vladimir Ivanov added a comment -

          The output with -Xtrace option is (last lines):
          ====================
          <...>
          EM: compile start:[JET_DPGO n=1009] java/io/PrintStream::flush()V
          EM: compile done:[JET_DPGO n=1009: OK] java/io/PrintStream::flush()V
          Compiled method java/io/PrintStream.flush()V, entry 0x2aaa2acff0
          compile_jit_a_method java/io/FilterOutputStream.flush()V
          EM: compile start:[JET_DPGO n=1010] java/io/FilterOutputStream::flush()V
          EM: compile done:[JET_DPGO n=1010: OK] java/io/FilterOutputStream::flush()V
          Compiled method java/io/FilterOutputStream.flush()V, entry 0x2aaa2ad2d0
          compile_jit_a_method java/io/OutputStream.flush()V
          EM: compile start:[JET_DPGO n=1011] java/io/OutputStream::flush()V
          EM: compile done:[JET_DPGO n=1011: OK] java/io/OutputStream::flush()V
          Compiled method java/io/OutputStream.flush()V, entry 0x2aaa2ad3c0
          compile_jit_a_method stress/Mix.allocate_chunk(I)Ljava/lang/Object;
          EM: compile start:[JET_DPGO n=1012] stress/Mix::allocate_chunk(I)Ljava/lang/Object;
          EM: compile done:[JET_DPGO n=1012: OK] stress/Mix::allocate_chunk(I)Ljava/lang/Object;
          Compiled method stress/Mix.allocate_chunk(I)Ljava/lang/Object;, entry 0x2aaa2b73a0
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          gc_alloc: 100016
          get next chunk: pinned_areas_pos = 1 pinned_areas.size() = 0
          starting gc = COPY
          limits 0x0x2a98b3f3b0 0x0x2a9980bffc
          stop_the_world_root_set_enumeration()
          Start thread suspension
          Thread suspension time: 49 mksec
          Clearing mark bits
          Segmentation fault

          Show
          Vladimir Ivanov added a comment - The output with -Xtrace option is (last lines): ==================== <...> EM: compile start: [JET_DPGO n=1009] java/io/PrintStream::flush()V EM: compile done: [JET_DPGO n=1009: OK] java/io/PrintStream::flush()V Compiled method java/io/PrintStream.flush()V, entry 0x2aaa2acff0 compile_jit_a_method java/io/FilterOutputStream.flush()V EM: compile start: [JET_DPGO n=1010] java/io/FilterOutputStream::flush()V EM: compile done: [JET_DPGO n=1010: OK] java/io/FilterOutputStream::flush()V Compiled method java/io/FilterOutputStream.flush()V, entry 0x2aaa2ad2d0 compile_jit_a_method java/io/OutputStream.flush()V EM: compile start: [JET_DPGO n=1011] java/io/OutputStream::flush()V EM: compile done: [JET_DPGO n=1011: OK] java/io/OutputStream::flush()V Compiled method java/io/OutputStream.flush()V, entry 0x2aaa2ad3c0 compile_jit_a_method stress/Mix.allocate_chunk(I)Ljava/lang/Object; EM: compile start: [JET_DPGO n=1012] stress/Mix::allocate_chunk(I)Ljava/lang/Object; EM: compile done: [JET_DPGO n=1012: OK] stress/Mix::allocate_chunk(I)Ljava/lang/Object; Compiled method stress/Mix.allocate_chunk(I)Ljava/lang/Object;, entry 0x2aaa2b73a0 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 gc_alloc: 100016 get next chunk: pinned_areas_pos = 1 pinned_areas.size() = 0 starting gc = COPY limits 0x0x2a98b3f3b0 0x0x2a9980bffc stop_the_world_root_set_enumeration() Start thread suspension Thread suspension time: 49 mksec Clearing mark bits Segmentation fault
          Hide
          Vladimir Ivanov added a comment -

          The next run print more info:
          vivanov1@nstdrlel18:/export/users/viv/trunk/cc/projects/drlvm/trunk/build/lnx_em64t_gcc_debug/semis/vm/_smoke.tests/classes> ../../../../deploy/jdk/jre/bin/java -cp . stress.Mix
          Co............................................................uccd+saeuccd+saeuccd+lsaleuccdjava: /export/users/viv/trunk/cc/projects/drlvm/trunk/vm/vmcore/include/object_layout.h:107: VTable* ManagedObject::vt(): Assertion `vt_offset' failed.
          Fatal error
          Segmentation fault

          Show
          Vladimir Ivanov added a comment - The next run print more info: vivanov1@nstdrlel18:/export/users/viv/trunk/cc/projects/drlvm/trunk/build/lnx_em64t_gcc_debug/semis/vm/_smoke.tests/classes> ../../../../deploy/jdk/jre/bin/java -cp . stress.Mix Co............................................................uccd+saeuccd+saeuccd+lsaleuccdjava: /export/users/viv/trunk/cc/projects/drlvm/trunk/vm/vmcore/include/object_layout.h:107: VTable* ManagedObject::vt(): Assertion `vt_offset' failed. Fatal error Segmentation fault

            People

            • Assignee:
              Xiao-Feng Li
              Reporter:
              Vladimir Ivanov
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development