MINA
  1. MINA
  2. DIRMINA-601

Add sendfile support to transport-apr

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-M2
    • Fix Version/s: 2.0.8
    • Component/s: Core, Transport
    • Labels:
      None

      Description

      I've take a shot at adding support for sendfile to transport-apr. I've had to make several changes because in the current code FileRegion is NIO specific because of it's reliance on FileChannel. I've made the following changes:

      I've created an interface called SendableFile to take the place of using FileChannel in FileRegion. I've also created two implementations of this interface, AprSendableFile and NioSendableFile. The method of interest is send(long position, long length) which is used to send the file. Notice it doesn't take an IoSession or a Channel as a parameter. I tried using generics to include the proper subclass of IoSession but that didn't work out very well.

      There are still a couple of problems that need to be dealt with. First and foremost is that AbstractIoSession will create instances of DefaultFileRegion in the write method when passed either a File or a FileChannel. This causes a problem because FileRegion is no longer NIO specific. I don't have a solution right now other than removing the code.

      The other problem is with FileRegionWriteFilter (and unit test), again because FileRegion is NIO specific and exposes a FileChannel.

      I'm hoping somebody can take a look at the code and provide some feedback if they believe this is headed in the right direction or have a better idea. I'll attach the patches shortly.

        Activity

        Emmanuel Lecharny made changes -
        Fix Version/s 2.0.8 [ 12323342 ]
        Fix Version/s 2.0.6 [ 12316652 ]
        Julien Vermillard made changes -
        Fix Version/s 2.0.6 [ 12316652 ]
        Fix Version/s 3.0.0-M1 [ 12313531 ]
        Emmanuel Lecharny made changes -
        Fix Version/s 3.0.0-M1 [ 12313531 ]
        Hide
        Emmanuel Lecharny added a comment -

        Postponed

        Show
        Emmanuel Lecharny added a comment - Postponed
        Geoff Cadien made changes -
        Attachment sendfile-patches.zip [ 12383497 ]
        Geoff Cadien made changes -
        Attachment sendfile-patches.zip [ 12383876 ]
        Hide
        Geoff Cadien added a comment -

        Added patch for NioProcessor to use SendableFile

        Show
        Geoff Cadien added a comment - Added patch for NioProcessor to use SendableFile
        Hide
        Geoff Cadien added a comment -

        Mike, You are correct that the ability to reuse a FileChannel is lost. I'm not sure open file descriptors is really that big of an issue. If your handling lots of open connections my guess is you'll saturate the network before you have problems with too many open files. org.apache.tomcat.jni.File doesn't have a constructor, just a bunch of static methods. The return value form File.open() is a long which I believe is a pointer to a struct which contains the fd and not the fd itself. Now I suppose if we added IoSession to the SendableFile.send() method you could keep the SendableFile around and reuse it lowering the number of open file descriptors.

        You're correct I did forget to add the patch to NioProcessor. I add that shortly.

        Show
        Geoff Cadien added a comment - Mike, You are correct that the ability to reuse a FileChannel is lost. I'm not sure open file descriptors is really that big of an issue. If your handling lots of open connections my guess is you'll saturate the network before you have problems with too many open files. org.apache.tomcat.jni.File doesn't have a constructor, just a bunch of static methods. The return value form File.open() is a long which I believe is a pointer to a struct which contains the fd and not the fd itself. Now I suppose if we added IoSession to the SendableFile.send() method you could keep the SendableFile around and reuse it lowering the number of open file descriptors. You're correct I did forget to add the patch to NioProcessor. I add that shortly.
        Hide
        Mike Heath added a comment -

        I see a number of difficulties with this situation. With the existing code, it is possible to use a previously open FileChannel. This makes it possible to serve a few files to many IoSessions without opening lots of file descriptors (which can be a limited resource in highly scalable applications.) With the provided patch, this is not possible.

        With APR, would it be possible to create a org.apache.tomcat.jni.File with a java.io.FileDescriptor. I used a java.io.FileDescriptor for the asynchronous File I/O stuff I did with POSIX AIO a year or so ago and it worked well. If so, the user could still send an open FileChannel in APR and we wouldn't have to open the file ourselves.

        Geoff have you patched the existing NIO IoProcessor code? I didn't see any of that in the attached patch.

        Show
        Mike Heath added a comment - I see a number of difficulties with this situation. With the existing code, it is possible to use a previously open FileChannel. This makes it possible to serve a few files to many IoSessions without opening lots of file descriptors (which can be a limited resource in highly scalable applications.) With the provided patch, this is not possible. With APR, would it be possible to create a org.apache.tomcat.jni.File with a java.io.FileDescriptor. I used a java.io.FileDescriptor for the asynchronous File I/O stuff I did with POSIX AIO a year or so ago and it worked well. If so, the user could still send an open FileChannel in APR and we wouldn't have to open the file ourselves. Geoff have you patched the existing NIO IoProcessor code? I didn't see any of that in the attached patch.
        Hide
        Geoff Cadien added a comment -

        I agree. That is exactly what I have done in my code. I have a factory to produce the correct type of SendableFile, using the type of IoSession as discriminator. It might be nice to have that functionality in MINA to avoid everyone having to roll their own.

        Show
        Geoff Cadien added a comment - I agree. That is exactly what I have done in my code. I have a factory to produce the correct type of SendableFile, using the type of IoSession as discriminator. It might be nice to have that functionality in MINA to avoid everyone having to roll their own.
        Hide
        Julien Vermillard added a comment -

        Nice patch !
        I think your SendableFile is a good abstraction for providing sendFile() implementation for transports.

        As I understand if you want to send a file using sendFile of an IoService, you need to instantiate the good SendableFile concrete class ? like AprSendable file if you use an AprIoacceptor.

        We need to be able to change from Nio IoService to Apr based IoService without changing the code.
        So I think it miss a per IoService SendableFile factory for creating SendableFile independently of the used transport.

        WDYT ?

        Show
        Julien Vermillard added a comment - Nice patch ! I think your SendableFile is a good abstraction for providing sendFile() implementation for transports. As I understand if you want to send a file using sendFile of an IoService, you need to instantiate the good SendableFile concrete class ? like AprSendable file if you use an AprIoacceptor. We need to be able to change from Nio IoService to Apr based IoService without changing the code. So I think it miss a per IoService SendableFile factory for creating SendableFile independently of the used transport. WDYT ?
        Geoff Cadien made changes -
        Attachment sendfile-patches.zip [ 12383476 ]
        Geoff Cadien made changes -
        Attachment sendfile-patches.zip [ 12383497 ]
        Hide
        Geoff Cadien added a comment -

        Forgot one patch

        Show
        Geoff Cadien added a comment - Forgot one patch
        Geoff Cadien made changes -
        Field Original Value New Value
        Attachment sendfile-patches.zip [ 12383476 ]
        Hide
        Geoff Cadien added a comment -

        I've attached the new classes/interfaces and diffs to exising classes to implement sendifle using the apr transport.

        Show
        Geoff Cadien added a comment - I've attached the new classes/interfaces and diffs to exising classes to implement sendifle using the apr transport.
        Geoff Cadien created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Geoff Cadien
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development