Directory ApacheDS
  1. Directory ApacheDS
  2. DIRSERVER-465

Make partitions nestable: remove the nexus singleton!

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.1.0
    • Component/s: core
    • Labels:
      None

      Description

      I just got a great idea but it might be a little crazy. There is a problem with the current model where there can only be one (singleton) nexus way at the top of a system. This severly limits how the namespace can be partitioned. Because of BackingStore operation routing concerns to ContextPartitions, one cannot have two ContextPartitions having a suffix overlap. Basically ContextPartitions with suffixes like so are not allowed:

      ..o CP1 suffix is dc=apache,dc=org
      ..o CP2 suffix is ou=people,dc=apache,dc=org

      Here's how the tree might look:

      .............................[RootNexus]
      ................................/...\
      ............................[CP1]...[CP2]

      The suffix of CP1 overlaps the suffix of CP2. Basically requests recieved under the CP2 suffix base have a tough time determining where they should route calls. To avoid this confusion there is the restriction mentioned above.

      What if had a very special kind of nexus that was not a singleton and had a suffix associated with it? Furthermore this nexus can contain entries off of that suffix as well instead of just delagating their storage to partitions it bridges. This nexus could then eliminate the routing problem and remove the restriction. For the time being lets call this a ContextNexus. So we could have the following configuration:

      ..o CN1 suffix is dc=apache,dc=org
      ..o CP2 suffix is ou=people,dc=apache,dc=org

      Here's how the tree might look:

      .............................[RootNexus]
      ..................................|
      ................................[CN1]
      ..................................|
      ................................[CP2]

      Here in this case entries like ou=groups,dc=apache,dc=apache and its contents would be stored in CN1 and so would the suffix dc=apache,dc=org. Now because of this extra level of routing decisions for routing cannot get confusing. We can then partition the namespace in any manner we see fit with minimal cost to performance.

        Issue Links

          Activity

          Alex Karasulu created issue -
          Hide
          Alex Karasulu added a comment -

          fixed formatting using dots cuz spaces are parsed out it seems

          Show
          Alex Karasulu added a comment - fixed formatting using dots cuz spaces are parsed out it seems
          Alex Karasulu made changes -
          Field Original Value New Value
          Description I just got a great idea but it might be a little crazy. There is a problem with the current model where there can only be one (singleton) nexus way at the top of a system. This severly limits how the namespace can be partitioned. Because of BackingStore operation routing concerns to ContextPartitions, one cannot have two ContextPartitions having a suffix overlap. Basically ContextPartitions with suffixes like so are not allowed:

          .o CP1 suffix is dc=apache,dc=org
          .o CP2 suffix is ou=people,dc=apache,dc=org

          Here's how the tree might look:

          . [RootNexus]
          . / \
          . [CP1] [CP2]
          .

          The suffix of CP1 overlaps the suffix of CP2. Basically requests recieved under the CP2 suffix base have a tough time determining where they should route calls. To avoid this confusion there is the restriction mentioned above.

          What if had a very special kind of nexus that was not a singleton and had a suffix associated with it? Furthermore this nexus can contain entries off of that suffix as well instead of just delagating their storage to partitions it bridges. This nexus could then eliminate the routing problem and remove the restriction. For the time being lets call this a ContextNexus. So we could have the following configuration:

           o CN1 suffix is dc=apache,dc=org
           o CP2 suffix is ou=people,dc=apache,dc=org

          Here's how the tree might look:

          . [RootNexus]
          . |
          . [CN1]
          . |
          . [CP2]

          Here in this case entries like ou=groups,dc=apache,dc=apache and its contents would be stored in CN1 and so would the suffix dc=apache,dc=org. Now because of this extra level of routing decisions for routing cannot get confusing. We can then partition the namespace in any manner we see fit with minimal cost to performance.

          I just got a great idea but it might be a little crazy. There is a problem with the current model where there can only be one (singleton) nexus way at the top of a system. This severly limits how the namespace can be partitioned. Because of BackingStore operation routing concerns to ContextPartitions, one cannot have two ContextPartitions having a suffix overlap. Basically ContextPartitions with suffixes like so are not allowed:

          ..o CP1 suffix is dc=apache,dc=org
          ..o CP2 suffix is ou=people,dc=apache,dc=org

          Here's how the tree might look:

          .............................[RootNexus]
          ................................/...\
          ............................[CP1]...[CP2]

          The suffix of CP1 overlaps the suffix of CP2. Basically requests recieved under the CP2 suffix base have a tough time determining where they should route calls. To avoid this confusion there is the restriction mentioned above.

          What if had a very special kind of nexus that was not a singleton and had a suffix associated with it? Furthermore this nexus can contain entries off of that suffix as well instead of just delagating their storage to partitions it bridges. This nexus could then eliminate the routing problem and remove the restriction. For the time being lets call this a ContextNexus. So we could have the following configuration:

          ..o CN1 suffix is dc=apache,dc=org
          ..o CP2 suffix is ou=people,dc=apache,dc=org

          Here's how the tree might look:

          .............................[RootNexus]
          ..................................|
          ................................[CN1]
          ..................................|
          ................................[CP2]

          Here in this case entries like ou=groups,dc=apache,dc=apache and its contents would be stored in CN1 and so would the suffix dc=apache,dc=org. Now because of this extra level of routing decisions for routing cannot get confusing. We can then partition the namespace in any manner we see fit with minimal cost to performance.

          Alex Karasulu made changes -
          Priority Major [ 3 ] Minor [ 4 ]
          Alex Karasulu made changes -
          Key DIREVE-23 DIRSERVER-465
          Component/s jndi-provider [ 11088 ]
          Project Directory Server [ 10516 ] Directory ApacheDS [ 12310260 ]
          Alex Karasulu made changes -
          Component/s core [ 12310713 ]
          Ersin Er made changes -
          Link This issue relates to DIRSERVER-617 [ DIRSERVER-617 ]
          Hide
          Alex Karasulu added a comment -

          With a single nexus that itself cannot store entries the server is severely constrained.

          (1) We cannot have subentries under the RootDSE so the RootDSE cannot be an administrative point.
          (2) The RootDSE must be read only and cannot be dynamically changed even by the admin.
          (3) A parent with millions of children who themselves have millions of children cannot be partitioned separately from it's children which leads to performance and capacity issues.
          (4) Aliases are not allowed across partitions and so linking entries in different domains via aliases is not possible and may be required to share users or groups across domains.
          (5) The number of out of the box partitions is growing (ou=system, ou=schema); What's next?
          (6) The alias problem can be solved by a nexus that can store entries; it can consolidate and track alias indices across nested partitions.

          This is something that should be addressed before the bigbang completes.

          Show
          Alex Karasulu added a comment - With a single nexus that itself cannot store entries the server is severely constrained. (1) We cannot have subentries under the RootDSE so the RootDSE cannot be an administrative point. (2) The RootDSE must be read only and cannot be dynamically changed even by the admin. (3) A parent with millions of children who themselves have millions of children cannot be partitioned separately from it's children which leads to performance and capacity issues. (4) Aliases are not allowed across partitions and so linking entries in different domains via aliases is not possible and may be required to share users or groups across domains. (5) The number of out of the box partitions is growing (ou=system, ou=schema); What's next? (6) The alias problem can be solved by a nexus that can store entries; it can consolidate and track alias indices across nested partitions. This is something that should be addressed before the bigbang completes.
          Hide
          Alex Karasulu added a comment -

          Putting this on the agenda and roadmap.

          Show
          Alex Karasulu added a comment - Putting this on the agenda and roadmap.
          Alex Karasulu made changes -
          Fix Version/s bigbang [ 12312809 ]
          Hide
          Alex Karasulu added a comment -

          This is critical for ease of use.

          Show
          Alex Karasulu added a comment - This is critical for ease of use.
          Alex Karasulu made changes -
          Priority Minor [ 4 ] Critical [ 2 ]
          Hide
          Alex Karasulu added a comment -

          making summary more succinct and clear

          Show
          Alex Karasulu added a comment - making summary more succinct and clear
          Alex Karasulu made changes -
          Summary Use multiple PartitionNexus' for layered partitioning Make partitions nestable: remove the nexus singleton!
          Alex Karasulu made changes -
          Priority Critical [ 2 ] Major [ 3 ]
          Hide
          Emmanuel Lecharny added a comment -

          Has to be done.

          Show
          Emmanuel Lecharny added a comment - Has to be done.
          Emmanuel Lecharny made changes -
          Fix Version/s 1.5.3 [ 12312693 ]
          Fix Version/s bigbang [ 12312809 ]
          Hide
          Alex Karasulu added a comment -

          see roadmap

          Show
          Alex Karasulu added a comment - see roadmap
          Alex Karasulu made changes -
          Fix Version/s 1.5.3 [ 12312693 ]
          Fix Version/s 1.5.4 [ 12313147 ]
          Hide
          Alex Karasulu added a comment -

          Not on agenda until 2.5.0.

          Show
          Alex Karasulu added a comment - Not on agenda until 2.5.0.
          Alex Karasulu made changes -
          Fix Version/s 2.5.0 [ 12313271 ]
          Fix Version/s 1.5.4 [ 12313147 ]

            People

            • Assignee:
              Alex Karasulu
              Reporter:
              Alex Karasulu
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:

                Development