Directory ApacheDS
  1. Directory ApacheDS
  2. DIRSERVER-612

apacheds-tools dump command does NOT generate the correct base 64 output for binary attributes in LDIF dump

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0-RC2
    • Fix Version/s: 1.0-RC4, 1.5.0
    • Component/s: tools
    • Labels:
      None

      Description

      The dump command which produces LDIF dumps of partitions is not generating the base64 encoded values for binary attributes. It seems to be printing just the byte[] which shows the id of the object in the jvm instead of the base64 encoded value.

      1. ads-1.0RC4-DIRSERVER-612-apacheds.patch
        2 kB
        Pierre-Arnaud Marcelot
      2. ads-1.0RC4-DIRSERVER-612-shared.patch
        9 kB
        Pierre-Arnaud Marcelot
      3. ads-1.1.0-DIRSERVER-612.patch
        12 kB
        Pierre-Arnaud Marcelot
      4. ADS-1.1.0-2006-08-18.patch
        8 kB
        Pierre-Arnaud Marcelot
      5. ADS-1.1.0-2006-08-18_SecondEdition.patch
        8 kB
        Pierre-Arnaud Marcelot
      6. ADS-1.0-RC4-2006-08-18_apacheds.patch
        3 kB
        Pierre-Arnaud Marcelot
      7. ADS-1.0-RC4-2006-08-18_shared.patch
        5 kB
        Pierre-Arnaud Marcelot

        Activity

        Hide
        Pierre-Arnaud Marcelot added a comment -

        Here is a patch for this issue.

        Dump now generates base64 encoded for binary attributes and also text attributes that need to be base64 encoded (because they contain special characters, such as accent for example).

        The part comes in two parts :

        • One for apache/server-tools
        • One for shared/ldap
          (I couldn't managed to generate a global patch, 'svn diff' at root directory didn't give any difference... Strange...)

        This post contains the first part. It must be patched for /apacheds

        Show
        Pierre-Arnaud Marcelot added a comment - Here is a patch for this issue. Dump now generates base64 encoded for binary attributes and also text attributes that need to be base64 encoded (because they contain special characters, such as accent for example). The part comes in two parts : One for apache/server-tools One for shared/ldap (I couldn't managed to generate a global patch, 'svn diff' at root directory didn't give any difference... Strange...) This post contains the first part. It must be patched for /apacheds
        Hide
        Pierre-Arnaud Marcelot added a comment -

        Here is the second part.

        It must be patched from /shared.

        I added a new class LdifUtils containing a method verifying that a String is LDIF safe.
        I also implemented a TestCase for this new class and method.

        I use this method to check if encoded is needed for text attributes in the Dump Command.

        Show
        Pierre-Arnaud Marcelot added a comment - Here is the second part. It must be patched from /shared. I added a new class LdifUtils containing a method verifying that a String is LDIF safe. I also implemented a TestCase for this new class and method. I use this method to check if encoded is needed for text attributes in the Dump Command.
        Hide
        Pierre-Arnaud Marcelot added a comment -

        Last file...

        Here is a patch for the current trunks for v1.1.0.

        It must be applied from the root directory.

        Show
        Pierre-Arnaud Marcelot added a comment - Last file... Here is a patch for the current trunks for v1.1.0. It must be applied from the root directory.
        Hide
        Alex Karasulu added a comment -

        Anybody apply your patches yet? Looks like it's up for grabs. I'll take care of this one.

        Show
        Alex Karasulu added a comment - Anybody apply your patches yet? Looks like it's up for grabs. I'll take care of this one.
        Hide
        Alex Karasulu added a comment -

        I applied your patches 432490, 432491 for 1.0 and 432492 for the trunks for 1.1.

        THere is however a big problem. I noticed this after I applied the patches. I can roll back but instead I can take corrective patches from you. Here's what a dump now looks like:

        #---------------------

        1. Entry: 23
          #---------------------

        dn: uid=user.22,dc=example,dc=com
        pager: Attribute id : 'pager', Values : ['515-711-8297']
        employeenumber: Attribute id : 'employeenumber', Values : ['22']
        objectclass: Attribute id : 'objectclass', Values : ['person', 'organizationalPerson', 'inetOrgPerson']
        mobile: Attribute id : 'mobile', Values : ['338-209-4979']
        telephonenumber: Attribute id : 'telephonenumber', Values : ['619-921-3217']
        street: Attribute id : 'street', Values : ['76697 Ash Street']
        userpassword: Attribute id : 'userpassword', Values : ['cGFzc3dvcmQ=']
        uid: Attribute id : 'uid', Values : ['user.22']
        postalcode: Attribute id : 'postalcode', Values : ['09762']
        givenname: Attribute id : 'givenname', Values : ['Abigael']
        mail: Attribute id : 'mail', Values : ['user.22@cs.hacettepe.edu.tr']
        l: Attribute id : 'l', Values : ['Dothan']
        sn: Attribute id : 'sn', Values : ['Abel']
        cn: Attribute id : 'cn', Values : ['Abigael Abel']
        postaladdress: Attribute id : 'postaladdress', Values : ['Abigael Abel$76697 Ash Street$Dothan, CO 09762']
        description: Attribute id : 'description', Values : ['This is the description for Abigael Abel.']
        initials: Attribute id : 'initials', Values : ['AA']
        homephone: Attribute id : 'homephone', Values : ['490-458-8467']
        st: Attribute id : 'st', Values : ['CO']

        Seems like we're using toString() on an attribute object instead of on it's value. The good news is the base64 issue is gone.

        Can you supply a patch to fix this problem?

        Thanks!

        Show
        Alex Karasulu added a comment - I applied your patches 432490, 432491 for 1.0 and 432492 for the trunks for 1.1. THere is however a big problem. I noticed this after I applied the patches. I can roll back but instead I can take corrective patches from you. Here's what a dump now looks like: #--------------------- Entry: 23 #--------------------- dn: uid=user.22,dc=example,dc=com pager: Attribute id : 'pager', Values : ['515-711-8297'] employeenumber: Attribute id : 'employeenumber', Values : ['22'] objectclass: Attribute id : 'objectclass', Values : ['person', 'organizationalPerson', 'inetOrgPerson'] mobile: Attribute id : 'mobile', Values : ['338-209-4979'] telephonenumber: Attribute id : 'telephonenumber', Values : ['619-921-3217'] street: Attribute id : 'street', Values : ['76697 Ash Street'] userpassword: Attribute id : 'userpassword', Values : ['cGFzc3dvcmQ='] uid: Attribute id : 'uid', Values : ['user.22'] postalcode: Attribute id : 'postalcode', Values : ['09762'] givenname: Attribute id : 'givenname', Values : ['Abigael'] mail: Attribute id : 'mail', Values : ['user.22@cs.hacettepe.edu.tr'] l: Attribute id : 'l', Values : ['Dothan'] sn: Attribute id : 'sn', Values : ['Abel'] cn: Attribute id : 'cn', Values : ['Abigael Abel'] postaladdress: Attribute id : 'postaladdress', Values : ['Abigael Abel$76697 Ash Street$Dothan, CO 09762'] description: Attribute id : 'description', Values : ['This is the description for Abigael Abel.'] initials: Attribute id : 'initials', Values : ['AA'] homephone: Attribute id : 'homephone', Values : ['490-458-8467'] st: Attribute id : 'st', Values : ['CO'] Seems like we're using toString() on an attribute object instead of on it's value. The good news is the base64 issue is gone. Can you supply a patch to fix this problem? Thanks!
        Hide
        Alex Karasulu added a comment -

        It's pretty important that we make sure this dump tool works right before the next release. I'm raisint this to critical.

        Show
        Alex Karasulu added a comment - It's pretty important that we make sure this dump tool works right before the next release. I'm raisint this to critical.
        Hide
        Pierre-Arnaud Marcelot added a comment -

        No problem Alex.

        I'm now working on that.

        It seems that the toString method I used has been changed in the merge of the optimization branch. This method doesn't print LDIF anymore.

        I send the patch as soon as possible.

        Show
        Pierre-Arnaud Marcelot added a comment - No problem Alex. I'm now working on that. It seems that the toString method I used has been changed in the merge of the optimization branch. This method doesn't print LDIF anymore. I send the patch as soon as possible.
        Hide
        Pierre-Arnaud Marcelot added a comment -

        Here's the patch for Trunks.

        The output is now ok.

        I also corrected a possible bug. The DN is now encoded to Base64 if it needs to. It was printed as is before.

        #---------------------

        1. Entry: 138
          #---------------------

        dn: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ==
        cn:: TMOpY2hhcm55
        objectclass: person
        objectclass: top

        Moreover, lines are now folded every 80 characters.

        #---------------------

        1. Entry: 92
          #---------------------

        dn: cn=newperson, dc=example,dc=com
        jpegphoto:: /9j/4AAQSkZJRgABAQEASABIAAD//gAoRmlsZSB3cml0dGVuIGJ5IEFkb2JlIFBob3Rv
        vc2hvcKggNC4wAA3/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIx
        wcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyM
        jIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAMABAADASIAAhEBAxEB/8QAHwAAAQUB
        AQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE
        [...]

        Show
        Pierre-Arnaud Marcelot added a comment - Here's the patch for Trunks. The output is now ok. I also corrected a possible bug. The DN is now encoded to Base64 if it needs to. It was printed as is before. — #--------------------- Entry: 138 #--------------------- dn: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ== cn:: TMOpY2hhcm55 objectclass: person objectclass: top — Moreover, lines are now folded every 80 characters. — #--------------------- Entry: 92 #--------------------- dn: cn=newperson, dc=example,dc=com jpegphoto:: /9j/4AAQSkZJRgABAQEASABIAAD//gAoRmlsZSB3cml0dGVuIGJ5IEFkb2JlIFBob3Rv vc2hvcKggNC4wAA3/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIx wcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyM jIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAMABAADASIAAhEBAxEB/8QAHwAAAQUB AQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE [...] —
        Hide
        Pierre-Arnaud Marcelot added a comment -

        Just a small update to the patch since I noticed that if I forgot to add the additionnal : (Colon) next to the dn definition when the value is Base64 encoded. In my sample output above, "dn: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ==" should be "dn:: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ==".

        Sorry about that.

        Show
        Pierre-Arnaud Marcelot added a comment - Just a small update to the patch since I noticed that if I forgot to add the additionnal : (Colon) next to the dn definition when the value is Base64 encoded. In my sample output above, "dn: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ==" should be "dn:: Y249TMOpY2hhcm55LCBkYz1leGFtcGxlLGRjPWNvbQ==". Sorry about that.
        Hide
        Pierre-Arnaud Marcelot added a comment -

        The next two patches are for 1.0-RC4.

        This one has to be applied from /apacheds.

        Show
        Pierre-Arnaud Marcelot added a comment - The next two patches are for 1.0-RC4. This one has to be applied from /apacheds.
        Hide
        Pierre-Arnaud Marcelot added a comment -

        And this one has to be applied from /shared.

        Show
        Pierre-Arnaud Marcelot added a comment - And this one has to be applied from /shared.
        Hide
        Alex Karasulu added a comment -

        Applied second set of patches on 1.0 in versions 432590 and 432593 and for 1.1 trunks on version 432597.

        NOTE: I applied the SecondEdition patch instead of the first one for the 1.1 trunks. I guessed these patches were not additive.

        Show
        Alex Karasulu added a comment - Applied second set of patches on 1.0 in versions 432590 and 432593 and for 1.1 trunks on version 432597. NOTE: I applied the SecondEdition patch instead of the first one for the 1.1 trunks. I guessed these patches were not additive.
        Hide
        Alex Karasulu added a comment -

        reopen to edit fields

        Show
        Alex Karasulu added a comment - reopen to edit fields
        Hide
        Alex Karasulu added a comment -

        fixed in 1.1 as well

        Show
        Alex Karasulu added a comment - fixed in 1.1 as well
        Hide
        Alex Karasulu added a comment -

        closed now.

        Show
        Alex Karasulu added a comment - closed now.

          People

          • Assignee:
            Alex Karasulu
            Reporter:
            Alex Karasulu
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development