Directory ApacheDS
  1. Directory ApacheDS
  2. DIRSERVER-1719

[Perf] Modify the way we process entries to be returned

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-M6
    • Fix Version/s: 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now, we clone the entries we will return to the client just after having fetched them from the backend. This is necessary as we will remove and add some attributes and values from those entries, to comply with the user request.

      Another idea would be to compute the attributes (and values) to return, and when done, create a new entry with all those attributes.

      As a user rarely requires all the attributes (including the operational ones), this might save some processing, as in the current system we copy all the attributes, then we remove some of them.

      Even better, when the CoreSession is called from the LdapProtocol layer, we don't have to copy the attributes at all, we just have to write on the socket only the required attributes. This will be even faster than what we currently do.

        Activity

        Hide
        Emmanuel Lecharny added a comment -

        One other idea :

        • we could modify the way the filters so that they don't process Entry, but Attribute. Now when we loop on all the original entry (the one we fetched from the backend) attributes, we will go through all the filters to know if we should return the attribute, or discard it.
        • we will have to process the collective attribute in a bit different way, as they are added to the result entry, but basically, this is quite the same mechanism. We get back the added attributes, then we check against the other filters if the attribute should be returned or not
        • the ACI filter is also a bit different, because it checks the Values too. Here, the filter could return a copy of the modified Attribute, if and only if the Attribute has already been accepted by all the other filters (considering that the ACI filter will be the last one to proceed, this is ok).

        At the end, we don't have to clone the full entry, ot do we have to store intermediary modifications. We will just do that once, just when we generate the entry to return. If this is called by the network layer, we won't even create an entry, but serialize it directly into a ByteBuffer.

        Show
        Emmanuel Lecharny added a comment - One other idea : we could modify the way the filters so that they don't process Entry, but Attribute. Now when we loop on all the original entry (the one we fetched from the backend) attributes, we will go through all the filters to know if we should return the attribute, or discard it. we will have to process the collective attribute in a bit different way, as they are added to the result entry, but basically, this is quite the same mechanism. We get back the added attributes, then we check against the other filters if the attribute should be returned or not the ACI filter is also a bit different, because it checks the Values too. Here, the filter could return a copy of the modified Attribute, if and only if the Attribute has already been accepted by all the other filters (considering that the ACI filter will be the last one to proceed, this is ok). At the end, we don't have to clone the full entry, ot do we have to store intermediary modifications. We will just do that once, just when we generate the entry to return. If this is called by the network layer, we won't even create an entry, but serialize it directly into a ByteBuffer.
        Hide
        Emmanuel Lecharny added a comment -

        This was exactly what I had in mind : wrap the entr fetched from the backend, and store there the modification to be applied.

        The tricky part comes with the ACI which may hide some attribute's values, so the granularity is the value, not the attribute

        Show
        Emmanuel Lecharny added a comment - This was exactly what I had in mind : wrap the entr fetched from the backend, and store there the modification to be applied. The tricky part comes with the ACI which may hide some attribute's values, so the granularity is the value, not the attribute
        Hide
        Alex Karasulu added a comment -

        This is a great idea but we also have to take into account that some interceptors to produce their net effect have to alter the entry on it's way out the door. So this can be done but might have to be done using another more creative mechanism. I recommend using shadowing this way for example: at the bottom you have the copy pulled out from the DIB and around it you have a wrapper. Reads tunnel through and read what's at the bottom from the original only if nothing has changed. Changes are stored in the wrapper. The wrapper serves as a sort of modified value storage.

        This way what is bubbled up to the network layer is the original copy with the wrapper around it. The necessary information is read from it and returned to the client without copying while still having the effects of the interceptors incorporated into the returned results.

        WDYT?

        Show
        Alex Karasulu added a comment - This is a great idea but we also have to take into account that some interceptors to produce their net effect have to alter the entry on it's way out the door. So this can be done but might have to be done using another more creative mechanism. I recommend using shadowing this way for example: at the bottom you have the copy pulled out from the DIB and around it you have a wrapper. Reads tunnel through and read what's at the bottom from the original only if nothing has changed. Changes are stored in the wrapper. The wrapper serves as a sort of modified value storage. This way what is bubbled up to the network layer is the original copy with the wrapper around it. The necessary information is read from it and returned to the client without copying while still having the effects of the interceptors incorporated into the returned results. WDYT?

          People

          • Assignee:
            Unassigned
            Reporter:
            Emmanuel Lecharny
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development