Uploaded image for project: 'James Server'
  1. James Server
  2. JAMES-584

FileStreamStore diskspace leak for removed messages in file based spool under windows

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Labels:
      None
    • Environment:
      Win32

      Description

      Iwasa Kazmi reported this on server-user:


      Hi,

      I sent a message to a local user and the delivery was
      done successfully, but a FileStreamStore file is still
      in the `spool' directory.
      Is the FileStreamStore file used ?

      I'm using james-2.3.0rc1 on WinXP.

      I did the following:

      1. edit config.xml for disable server name detection.

      <servernames autodetect="false" autodetectIP="true">
      <servername>localhost</servername>
      </servernames>

      2. add user `iwasa'.

      3. send email to iwasa@localhost.

      Thank you,
      Iwasa Kazmi
      ------
      Confirmed by me here:
      ------
      FWIW I can reproduce the issue with current 2.3rc1 and current trunk.

      Config file as default for 2.3, repositories changed to file in trunk
      (we still have derby as default in trunk).

      I connect to remote manager, create an user. Connect to smtp server and
      send a message to that user.

      The XXX.Repository.FileStreamStore file is stuck there, in the spool.
      If I restart James it got cleaned by the cleaning procedure in
      AvalonMailRepository.

      This seems to be a bug. Every file that is spooled remain there until a
      restart. This is diskspace leak.

      I tried manually calling from all the places the
      ContainerUtil.dispose(mail) and set mail to null to be sure that the
      remove were called and I can see that remove is called twice on that
      file but it is not removed. It seems that a stream is still opened when
      we try to remove it and it's not removed.

      I've not investigated more on this. I only use filerepository in a small
      production server that I rarely check. Noel, Vincenzo, can you check
      what does it happen in the long term? Maybe this is a test for
      Postage... spool folder should increase it size in a linear way if this
      bug is there.
      ------
      This could be platform specific: I'm able to reproduce it under windows.

      This would also make sense because in unix os you can remove opened
      files and they are really removed when the file is closed while under
      windows an opened file cannot be deleted.

        Activity

        Hide
        bago Stefano Bagnara added a comment -

        Let's mark this fixed. Optimism never killed anyone

        Show
        bago Stefano Bagnara added a comment - Let's mark this fixed. Optimism never killed anyone
        Hide
        bago Stefano Bagnara added a comment -

        Iwasa Kazmi found another way to reproduce this issue:


        I inserted Redirect mailet in the "transport" processor.
        (see config.xml.diff)
        It duplicates a message for local-user before the RemoteDelivery runs.
        Then I sent a mail to the external server via james.
        FileStreamStore file in the spool was not deleted.


        Show
        bago Stefano Bagnara added a comment - Iwasa Kazmi found another way to reproduce this issue: I inserted Redirect mailet in the "transport" processor. (see config.xml.diff) It duplicates a message for local-user before the RemoteDelivery runs. Then I sent a mail to the external server via james. FileStreamStore file in the spool was not deleted.
        Hide
        bago Stefano Bagnara added a comment -

        Optimism didn't work yesterday... let's test it again...

        The main problem was that we were duplicating a Mail object using "new MailImpl(Mail)" that create a CoW proxy for the shared MimeMessage and then we were calling setMessage that replaced the previous message with no care of its "unreferencing".
        So, now, if we call setMessage and a message was already there then we dispose the previous one first, and then we set the new one.

        I also added a few mail.dispose to mailets that created local duplicates of serviced Mail and were not disposing it (AbstractRedirect, DSNBounce....).

        Doing all this stuff I also thought that if something gone wrong after the mail duplication and before the disposal we would have leaked, so I added a few try/finally code to make sure the dispose was called anyway.

        Show
        bago Stefano Bagnara added a comment - Optimism didn't work yesterday... let's test it again... The main problem was that we were duplicating a Mail object using "new MailImpl(Mail)" that create a CoW proxy for the shared MimeMessage and then we were calling setMessage that replaced the previous message with no care of its "unreferencing". So, now, if we call setMessage and a message was already there then we dispose the previous one first, and then we set the new one. I also added a few mail.dispose to mailets that created local duplicates of serviced Mail and were not disposing it (AbstractRedirect, DSNBounce....). Doing all this stuff I also thought that if something gone wrong after the mail duplication and before the disposal we would have leaked, so I added a few try/finally code to make sure the dispose was called anyway.
        Hide
        danny@apache.org Danny Angus added a comment -

        Closing issue fixed in released version.

        Show
        danny@apache.org Danny Angus added a comment - Closing issue fixed in released version.

          People

          • Assignee:
            bago Stefano Bagnara
            Reporter:
            bago Stefano Bagnara
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development