Ivy
  1. Ivy
  2. IVY-1075

Publish with SSH (sftp or ssh) prevents enclosing java process to terminate

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-RC1
    • Fix Version/s: 2.1.0-RC2
    • Component/s: None
    • Labels:
      None
    • Environment:

      buildr 1.3.4 running on jruby 1.2.0 with latest jsch-0.1.41 jar

      Description

      We try to add ivy support to buildr. Buildr comes with an AntWrap gem to call Ant targets from within a buildr script:
      1. using a standard publish via Antwrap using the same parameters that work in Ant
      2. the publish works all artifacts are copied and the buildr script continues till its end, but the buildr java process does not terminate!

      when publishing to a local repository the process terminates fine, so it has something to do with the SSH connection.
      using lsof and checking the open file handles the process has an open connection to the repository via SSH and some other open sockets (lsof output for process):
      [...]
      java 23589 klaas 31u IPv6 342503 TCP brackel.local:43473->argon:ssh (ESTABLISHED)
      java 23589 klaas 32u sock 0,4 342501 can't identify protocol
      java 23589 klaas 11u unix 0xf5744a80 342386 socket
      [...]

      I checked out the source but could not find the problem. My best guess is that the last SSH connection is cached within SshCache and never disconnected from Ivy.

      This is a blocker to change from ANT to Buildr, because the build scripts do not terminate normally so they cannot be used in the integration server.

      We have the same problem with an hanging jruby script using Ivy not via Ant but via the Ivy class to resolve projects. The Ivy class is instantiated from within JRuby and used to resolve ivy.xmls to get some information about projects. The script does not terminate after a resolve against a remote repository using SSH has been done.

        Issue Links

          Activity

          Hide
          Klaas Prause added a comment -

          I have a thread dump from the process:

          Full thread dump Java HotSpot(TM) Client VM (11.0-b15 mixed mode, sharing):

          "DestroyJavaVM" prio=10 tid=0xb4f99c00 nid=0x1a59 waiting on condition [0x00000000..0xb7df0060]
          java.lang.Thread.State: RUNNABLE

          "Connect thread ivy session" prio=10 tid=0xb4f9cc00 nid=0x1a71 runnable [0xb4bfe000..0xb4bfef20]
          java.lang.Thread.State: RUNNABLE
          at java.net.SocketInputStream.socketRead0(Native Method)
          at java.net.SocketInputStream.read(SocketInputStream.java:129)
          at com.jcraft.jsch.IO.getByte(IO.java:82)
          at com.jcraft.jsch.Session.read(Session.java:809)
          at com.jcraft.jsch.Session.run(Session.java:1193)
          at java.lang.Thread.run(Thread.java:619)

          "Low Memory Detector" daemon prio=10 tid=0xb5400c00 nid=0x1a5f runnable [0x00000000..0x00000000]
          java.lang.Thread.State: RUNNABLE

          "CompilerThread0" daemon prio=10 tid=0x09818000 nid=0x1a5e waiting on condition [0x00000000..0xb55e3a58]
          java.lang.Thread.State: RUNNABLE

          "Signal Dispatcher" daemon prio=10 tid=0x09816800 nid=0x1a5d waiting on condition [0x00000000..0x00000000]
          java.lang.Thread.State: RUNNABLE

          "Finalizer" daemon prio=10 tid=0x09810400 nid=0x1a5c in Object.wait() [0xb582b000..0xb582bea0]
          java.lang.Thread.State: WAITING (on object monitor)
          at java.lang.Object.wait(Native Method)

          • waiting on <0x733313b0> (a java.lang.ref.ReferenceQueue$Lock)
            at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
          • locked <0x733313b0> (a java.lang.ref.ReferenceQueue$Lock)
            at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
            at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

          "Reference Handler" daemon prio=10 tid=0x0980bc00 nid=0x1a5b in Object.wait() [0xb592c000..0xb592ce20]
          java.lang.Thread.State: WAITING (on object monitor)
          at java.lang.Object.wait(Native Method)

          • waiting on <0x73331438> (a java.lang.ref.Reference$Lock)
            at java.lang.Object.wait(Object.java:485)
            at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
          • locked <0x73331438> (a java.lang.ref.Reference$Lock)

          "VM Thread" prio=10 tid=0x0980a000 nid=0x1a5a runnable

          "VM Periodic Task Thread" prio=10 tid=0xb5402c00 nid=0x1a60 waiting on condition

          JNI global references: 935

          Heap
          def new generation total 2112K, used 1964K [0x70c80000, 0x70ec0000, 0x732f0000)
          eden space 1920K, 98% used [0x70c80000, 0x70e5ac68, 0x70e60000)
          from space 192K, 34% used [0x70e60000, 0x70e706f0, 0x70e90000)
          to space 192K, 0% used [0x70e90000, 0x70e90000, 0x70ec0000)
          tenured generation total 27400K, used 16688K [0x732f0000, 0x74db2000, 0x90080000)
          the space 27400K, 60% used [0x732f0000, 0x7433c338, 0x7433c400, 0x74db2000)
          compacting perm gen total 21504K, used 21328K [0x90080000, 0x91580000, 0x94080000)
          the space 21504K, 99% used [0x90080000, 0x91554200, 0x91554200, 0x91580000)
          ro space 8192K, 74% used [0x94080000, 0x94673488, 0x94673600, 0x94880000)
          rw space 12288K, 58% used [0x94880000, 0x94f8db20, 0x94f8dc00, 0x95480000)

          Show
          Klaas Prause added a comment - I have a thread dump from the process: Full thread dump Java HotSpot(TM) Client VM (11.0-b15 mixed mode, sharing): "DestroyJavaVM" prio=10 tid=0xb4f99c00 nid=0x1a59 waiting on condition [0x00000000..0xb7df0060] java.lang.Thread.State: RUNNABLE "Connect thread ivy session" prio=10 tid=0xb4f9cc00 nid=0x1a71 runnable [0xb4bfe000..0xb4bfef20] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at com.jcraft.jsch.IO.getByte(IO.java:82) at com.jcraft.jsch.Session.read(Session.java:809) at com.jcraft.jsch.Session.run(Session.java:1193) at java.lang.Thread.run(Thread.java:619) "Low Memory Detector" daemon prio=10 tid=0xb5400c00 nid=0x1a5f runnable [0x00000000..0x00000000] java.lang.Thread.State: RUNNABLE "CompilerThread0" daemon prio=10 tid=0x09818000 nid=0x1a5e waiting on condition [0x00000000..0xb55e3a58] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x09816800 nid=0x1a5d waiting on condition [0x00000000..0x00000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x09810400 nid=0x1a5c in Object.wait() [0xb582b000..0xb582bea0] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) waiting on <0x733313b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) locked <0x733313b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) "Reference Handler" daemon prio=10 tid=0x0980bc00 nid=0x1a5b in Object.wait() [0xb592c000..0xb592ce20] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) waiting on <0x73331438> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) locked <0x73331438> (a java.lang.ref.Reference$Lock) "VM Thread" prio=10 tid=0x0980a000 nid=0x1a5a runnable "VM Periodic Task Thread" prio=10 tid=0xb5402c00 nid=0x1a60 waiting on condition JNI global references: 935 Heap def new generation total 2112K, used 1964K [0x70c80000, 0x70ec0000, 0x732f0000) eden space 1920K, 98% used [0x70c80000, 0x70e5ac68, 0x70e60000) from space 192K, 34% used [0x70e60000, 0x70e706f0, 0x70e90000) to space 192K, 0% used [0x70e90000, 0x70e90000, 0x70ec0000) tenured generation total 27400K, used 16688K [0x732f0000, 0x74db2000, 0x90080000) the space 27400K, 60% used [0x732f0000, 0x7433c338, 0x7433c400, 0x74db2000) compacting perm gen total 21504K, used 21328K [0x90080000, 0x91580000, 0x94080000) the space 21504K, 99% used [0x90080000, 0x91554200, 0x91554200, 0x91580000) ro space 8192K, 74% used [0x94080000, 0x94673488, 0x94673600, 0x94880000) rw space 12288K, 58% used [0x94880000, 0x94f8db20, 0x94f8dc00, 0x95480000)
          Hide
          Klaas Prause added a comment -

          From the thread dump the operation blocks in JSch library. The Session class has one thread for the connection and blocks inside the run method trying to read bytes from the server. The read method is documented as blocking till bytes can be read, end of file is encountered or an exception is thrown.

          I cannot say if the Session from jsch is the problem or the way ivy uses the library. I do not understand the difference between normal use via Ant and the use via Buildr an Antwrap.

          Show
          Klaas Prause added a comment - From the thread dump the operation blocks in JSch library. The Session class has one thread for the connection and blocks inside the run method trying to read bytes from the server. The read method is documented as blocking till bytes can be read, end of file is encountered or an exception is thrown. I cannot say if the Session from jsch is the problem or the way ivy uses the library. I do not understand the difference between normal use via Ant and the use via Buildr an Antwrap.
          Hide
          Klaas Prause added a comment - - edited

          this issue describes the problem:
          http://svnkit.com/tracker/view.php?id=52

          when I am end the buildr process explicitly with "System.exit(0)" the open SSH connections are closed and the process returns.

          Maybe Ivy should provide an option to terminate the open SSH connections in the pool as implemented in SVNKit (SVNJschSession.shutdown()).

          Show
          Klaas Prause added a comment - - edited this issue describes the problem: http://svnkit.com/tracker/view.php?id=52 when I am end the buildr process explicitly with "System.exit(0)" the open SSH connections are closed and the process returns. Maybe Ivy should provide an option to terminate the open SSH connections in the pool as implemented in SVNKit (SVNJschSession.shutdown()).
          Hide
          Klaas Prause added a comment -

          not a bug

          Show
          Klaas Prause added a comment - not a bug
          Hide
          Maarten Coene added a comment -

          I think I was able to solve this problem without adding an extra Ant task to close the sessions.
          Could you give it a try (I cannot test it to see if it really works) and provide us your feedback?

          thanks,
          Maarten

          Show
          Maarten Coene added a comment - I think I was able to solve this problem without adding an extra Ant task to close the sessions. Could you give it a try (I cannot test it to see if it really works) and provide us your feedback? thanks, Maarten
          Hide
          Klaas Prause added a comment -

          fixed in trunk, thank you very much!

          Show
          Klaas Prause added a comment - fixed in trunk, thank you very much!

            People

            • Assignee:
              Maarten Coene
              Reporter:
              Klaas Prause
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development