James Server
  1. James Server
  2. JAMES-607

Rewrite MBoxMailRepository to use mstor

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 3.0-M1
    • Labels:
      None

      Description

      MBoxMailRepository should be rewritten to use mstor. Mstor provide a clean way to access mbox files etc

      1. mbox.diff
        39 kB
        Norman Maurer

        Activity

        Hide
        Norman Maurer added a comment -

        Here it is. Plz review

        Show
        Norman Maurer added a comment - Here it is. Plz review
        Hide
        Norman Maurer added a comment -

        Stefano point out that we don't need this class anymore after i did the work ;-P

        We can simply use the javamailstore which joachim develop and use mstor as javamailstore provider.

        Here is what you have todo to get it work:

        1. Get the javamailstore-mailrepository-5.jar from http://issues.apache.org/jira/browse/JAMES-461 and place it to /path/to/james/apps/james/SAR-INF/lib/

        2. Change the mbox config to:
        <inboxRepository>
        <repository destinationURL="mstor://var/mail/" postfix="/" type="MAIL"/>
        </inboxRepository>

        <repository class="org.apache.james.mailrepository.javamail.HashJavamailStoreMailRepository">
        <protocols>
        <protocol>mstor</protocol>
        </protocols>
        <types>
        <type>MAIL</type>
        </types>
        </repository>

        Thats all. I whould be +1 to include joachims work finally to james trunk

        Show
        Norman Maurer added a comment - Stefano point out that we don't need this class anymore after i did the work ;-P We can simply use the javamailstore which joachim develop and use mstor as javamailstore provider. Here is what you have todo to get it work: 1. Get the javamailstore-mailrepository-5.jar from http://issues.apache.org/jira/browse/JAMES-461 and place it to /path/to/james/apps/james/SAR-INF/lib/ 2. Change the mbox config to: <inboxRepository> <repository destinationURL="mstor://var/mail/" postfix="/" type="MAIL"/> </inboxRepository> <repository class="org.apache.james.mailrepository.javamail.HashJavamailStoreMailRepository"> <protocols> <protocol>mstor</protocol> </protocols> <types> <type>MAIL</type> </types> </repository> Thats all. I whould be +1 to include joachims work finally to james trunk
        Hide
        Noel J. Bergman added a comment -

        We should not be using JavaMail for storage except to interface to existing systems. JavaMail is intended to be a client-side API, and is not really suitable for performant work on the server-side.

        Show
        Noel J. Bergman added a comment - We should not be using JavaMail for storage except to interface to existing systems. JavaMail is intended to be a client-side API, and is not really suitable for performant work on the server-side.
        Hide
        Stefano Bagnara added a comment -

        From what I understood the mstor implementation (javamail store implementation) is much more performant than our custom mboxrepository. In fact mboxrepository had very big performance limit when used for reading and there was a lot of TODO in the list.

        Show
        Stefano Bagnara added a comment - From what I understood the mstor implementation (javamail store implementation) is much more performant than our custom mboxrepository. In fact mboxrepository had very big performance limit when used for reading and there was a lot of TODO in the list.
        Hide
        Noel J. Bergman added a comment -

        The question isn't the mbox code that we wrote vs mstor, which is apparently much faster (no surprise, actually). It is integrating mstor into a repository interface vs JavaMail. And for importing from MBOX or MailDir, it might be fine to use JavaMail. I'm refer to any general idea of using JavaMail for local storage.

        Show
        Noel J. Bergman added a comment - The question isn't the mbox code that we wrote vs mstor, which is apparently much faster (no surprise, actually). It is integrating mstor into a repository interface vs JavaMail. And for importing from MBOX or MailDir, it might be fine to use JavaMail. I'm refer to any general idea of using JavaMail for local storage.
        Hide
        Joachim Draeger added a comment -

        A note to mbox in general:
        In a medium-term James should support different feature-levels of repositories. The "client" generated permanent keys will never fit satisfyingly to a mbox backend.
        In the normal SMTP -> POP3 lifecycle no permanent keys are needed at all and mbox could be perfectly supported without any hash-sum workarounds.

        Joachim

        Show
        Joachim Draeger added a comment - A note to mbox in general: In a medium-term James should support different feature-levels of repositories. The "client" generated permanent keys will never fit satisfyingly to a mbox backend. In the normal SMTP -> POP3 lifecycle no permanent keys are needed at all and mbox could be perfectly supported without any hash-sum workarounds. Joachim
        Hide
        Joachim Draeger added a comment -

        I had no luck using mstor 0.9.9

        • it's not checksum save because it returns additional (mbox related) \r\n when fetching a message that it not the last one
        • it messes internal cache up, when reopening folder
          1. add 2 Messages
          2. delete and expunge 1. message, close folder
          3. open and close with expunge: the retaining message gets deleted because an unclean DELETED flag is
          assumed
        • metadata file doesn't get closed after save, so you can't delete it
          So IMHO it is unusable by now.

        But I have something positive to say, too:
        It's code looks quite decent, structured and clean
        And I was wrong saying that would be complex to add flag and uid support! It uses a xml metadata file along with the mbox file which already is able to store Flags and could be easily extended to support UIDs.
        BTW there don't seem to be any changes in trunk till 0.9.9

        I'l try to write unit tests and patches for what I've found and submit them to mstor but I wasn't able to fix everything by now. Anyway: very promising.

        Show
        Joachim Draeger added a comment - I had no luck using mstor 0.9.9 it's not checksum save because it returns additional (mbox related) \r\n when fetching a message that it not the last one it messes internal cache up, when reopening folder 1. add 2 Messages 2. delete and expunge 1. message, close folder 3. open and close with expunge: the retaining message gets deleted because an unclean DELETED flag is assumed metadata file doesn't get closed after save, so you can't delete it So IMHO it is unusable by now. But I have something positive to say, too: It's code looks quite decent, structured and clean And I was wrong saying that would be complex to add flag and uid support! It uses a xml metadata file along with the mbox file which already is able to store Flags and could be easily extended to support UIDs. BTW there don't seem to be any changes in trunk till 0.9.9 I'l try to write unit tests and patches for what I've found and submit them to mstor but I wasn't able to fix everything by now. Anyway: very promising.
        Hide
        Norman Maurer added a comment -

        Sad to hear that mstor not work like aspected by now.

        But anyway i think if we supply patches to the author he we probaly fix the bugs ( i hope so)

        Show
        Norman Maurer added a comment - Sad to hear that mstor not work like aspected by now. But anyway i think if we supply patches to the author he we probaly fix the bugs ( i hope so)
        Hide
        Joachim Draeger added a comment -

        I just created bug reports for the ones I found. But I fear there maybe more.

        1567158 Metadata file doesn't get closed after been saved
        http://sourceforge.net/tracker/index.php?func=detail&aid=1567158&group_id=114229&atid=667638

        1567139 Messages read from Folder contain additional \r\n
        http://sourceforge.net/tracker/index.php?func=detail&aid=1567139&group_id=114229&atid=667638

        1565328 messageCache out of date after reopen causes message loss
        http://sourceforge.net/tracker/index.php?func=detail&aid=1565328&group_id=114229&atid=667638

        Show
        Joachim Draeger added a comment - I just created bug reports for the ones I found. But I fear there maybe more. 1567158 Metadata file doesn't get closed after been saved http://sourceforge.net/tracker/index.php?func=detail&aid=1567158&group_id=114229&atid=667638 1567139 Messages read from Folder contain additional \r\n http://sourceforge.net/tracker/index.php?func=detail&aid=1567139&group_id=114229&atid=667638 1565328 messageCache out of date after reopen causes message loss http://sourceforge.net/tracker/index.php?func=detail&aid=1565328&group_id=114229&atid=667638
        Hide
        Norman Maurer added a comment -

        Move to trunk until the bugs are fixed

        Show
        Norman Maurer added a comment - Move to trunk until the bugs are fixed
        Hide
        Norman Maurer added a comment -

        I don't think we ever include this.. So close it

        Show
        Norman Maurer added a comment - I don't think we ever include this.. So close it

          People

          • Assignee:
            Norman Maurer
            Reporter:
            Norman Maurer
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development