Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.9.1.0
    • Component/s: Documentation
    • Labels:
      None
    1. DERBY-5442-3.zip
      9 kB
      Kim Haase
    2. DERBY-5442-3.stat
      0.1 kB
      Kim Haase
    3. DERBY-5442-3.diff
      3 kB
      Kim Haase
    4. DERBY-5442-2.zip
      19 kB
      Kim Haase
    5. DERBY-5442-2.diff
      17 kB
      Kim Haase
    6. DERBY-5442.zip
      18 kB
      Kim Haase
    7. DERBY-5442.stat
      0.3 kB
      Kim Haase
    8. DERBY-5442.diff
      12 kB
      Kim Haase

      Issue Links

        Activity

        Dag H. Wanvik created issue -
        Knut Anders Hatlen made changes -
        Field Original Value New Value
        Link This issue is related to DERBY-5465 [ DERBY-5465 ]
        Knut Anders Hatlen made changes -
        Link This issue is related to DERBY-5465 [ DERBY-5465 ]
        Kim Haase made changes -
        Link This issue relates to DERBY-5363 [ DERBY-5363 ]
        Hide
        Kim Haase added a comment -

        See release note for DERBY-5363.

        Show
        Kim Haase added a comment - See release note for DERBY-5363 .
        Kim Haase made changes -
        Assignee Kim Haase [ chaase3 ]
        Hide
        Kim Haase added a comment -

        It looks as if the default when you don't set the new property derby.storage.useDefaultFilePermissions is different from the behavior when you set it to either false or true. Am I understanding this correctly? I'm looking at the last version of the release note for DERBY-5363.

        If you don't set it (and you run using Java 7), the Derby Network Server limits access for Derby-created files and directories to the user account that booted the server.

        If you set it to true, the server uses whatever default file permissions the user has set for their system.

        If you set it to false, access is limited not only for files created by the Network Server, but also for embedded databases and for databases managed by servers which were started programmatically inside application code via the Derby API.

        So once you have set the property one way or the other, the way to return to the default behavior is to restart the server without setting the property.

        I'm looking into where this should be documented, other than the property documentation in the Reference Manual:

        Developer's Guide: the section "Configuring security in a client/server environment", which talks about operating system permissions for files. There should at least be a mention of the feature.

        Admin Guide: Maybe a new topic under "Derby Network Server advanced topics"? The feature is related to Network Server security. This new topic might be cross-referenced from "Starting the Network Server".

        If you have any other locations in mind, please let me know. I may find some more myself.

        Show
        Kim Haase added a comment - It looks as if the default when you don't set the new property derby.storage.useDefaultFilePermissions is different from the behavior when you set it to either false or true. Am I understanding this correctly? I'm looking at the last version of the release note for DERBY-5363 . If you don't set it (and you run using Java 7), the Derby Network Server limits access for Derby-created files and directories to the user account that booted the server. If you set it to true, the server uses whatever default file permissions the user has set for their system. If you set it to false, access is limited not only for files created by the Network Server, but also for embedded databases and for databases managed by servers which were started programmatically inside application code via the Derby API. So once you have set the property one way or the other, the way to return to the default behavior is to restart the server without setting the property. I'm looking into where this should be documented, other than the property documentation in the Reference Manual: Developer's Guide: the section "Configuring security in a client/server environment", which talks about operating system permissions for files. There should at least be a mention of the feature. Admin Guide: Maybe a new topic under "Derby Network Server advanced topics"? The feature is related to Network Server security. This new topic might be cross-referenced from "Starting the Network Server". If you have any other locations in mind, please let me know. I may find some more myself.
        Hide
        Dag H. Wanvik added a comment -

        I believe your understanding is correct, Kim, with the added proviso to your paragraph two that the server must be started from the command line for the (new) default behavior to apply. You do mention that in your paragraph four. If the server is started via the API, the default is as before, i.e. as if the the property had been set to true. It is correct that a restarted engine does not "remember" the property's setting from the previous boot, so that is in deed a way to fall back. In the case of server started from the command line, the old behavior can only be secured by setting the property to false.

        Show
        Dag H. Wanvik added a comment - I believe your understanding is correct, Kim, with the added proviso to your paragraph two that the server must be started from the command line for the (new) default behavior to apply. You do mention that in your paragraph four. If the server is started via the API, the default is as before, i.e. as if the the property had been set to true. It is correct that a restarted engine does not "remember" the property's setting from the previous boot, so that is in deed a way to fall back. In the case of server started from the command line, the old behavior can only be secured by setting the property to false.
        Hide
        Kim Haase added a comment -

        Thanks, Dag, for the clarifications. That helps. Though I am confused by this sentence: "In the case of server started from the command line, the old behavior can only be secured by setting the property to false." Isn't it setting the property to true that restores the old default behavior?

        Just a couple of housekeeping questions:

        Can the property be set on both a system-wide and a database-wide level? I'm guessing system-wide only, since it doesn't make much sense as a database property?

        Also, is the property static or dynamic?

        Show
        Kim Haase added a comment - Thanks, Dag, for the clarifications. That helps. Though I am confused by this sentence: "In the case of server started from the command line, the old behavior can only be secured by setting the property to false." Isn't it setting the property to true that restores the old default behavior? Just a couple of housekeeping questions: Can the property be set on both a system-wide and a database-wide level? I'm guessing system-wide only, since it doesn't make much sense as a database property? Also, is the property static or dynamic?
        Hide
        Kim Haase added a comment -

        On the hypothesis that the property is system-wide and static, I'm going ahead and filing a patch with new topics in the Reference Manual and Admin Guide, modifications to topics to point to the new topics, and the addition of some text in a logical place in the Developer's Guide (I hope).

        I'm sure these need some work – I look forward to feedback. Thanks in advance, Dag!

        Show
        Kim Haase added a comment - On the hypothesis that the property is system-wide and static, I'm going ahead and filing a patch with new topics in the Reference Manual and Admin Guide, modifications to topics to point to the new topics, and the addition of some text in a logical place in the Developer's Guide (I hope). I'm sure these need some work – I look forward to feedback. Thanks in advance, Dag!
        Kim Haase made changes -
        Attachment DERBY-5442.diff [ 12521183 ]
        Attachment DERBY-5442.stat [ 12521184 ]
        Attachment DERBY-5442.zip [ 12521185 ]
        Kim Haase made changes -
        Issue & fix info Patch Available [ 10102 ]
        Hide
        Dag H. Wanvik added a comment -

        Thanks Kim, you are right about the property value, my bad. Of course it should be true. It's system-wide only, yes. But if the application has the privilege to set a system property, it is dynamic, cf. logic in FileUtil#limitAccessToOwner, which calls PropertyUtil#getSystemProperty(STORAGE_USE_DEFAULT_FILE_PERMISSIONS). Changing the value dynamically is not recommended.

        Show
        Dag H. Wanvik added a comment - Thanks Kim, you are right about the property value, my bad. Of course it should be true . It's system-wide only, yes. But if the application has the privilege to set a system property, it is dynamic, cf. logic in FileUtil#limitAccessToOwner, which calls PropertyUtil#getSystemProperty(STORAGE_USE_DEFAULT_FILE_PERMISSIONS). Changing the value dynamically is not recommended.
        Hide
        Dag H. Wanvik added a comment -

        Patch comment:

        • Refman topic

        > derby.storage.useDefaultFilePermissions
        > Function
        >
        > If you run with Java SE 7, and if you start the Derby Network Server
        > from the command line, access to databases and to other Derby files is
        > by default restricted to the operating system account that started the
        > Network Server. File access is not restricted for embedded databases
        > or for databases managed by servers that are started programmatically
        > inside application code using the Derby API.

        "If you run with Java SE 7" -> "If you run with Java SE 7 or newer"

        I'd say "database files and other files created by Derby",
        i.e. derby.properties is a Derby file but not created by Derby and its
        access will not be modified.

        As for the logic here, I wonder if we should describe this in another
        way to make it less confusing, maybe a matrix?
        (sorry it doesn't look good in here but you get the idea

        File access determined by:

        E: access controlled entirely by OS environment of JVM, e.g. effective Unix umask or Window default permissions
        R: Derby restricts access to the operating system account that started the JVM

        <= Java 6

        Server from cmd line Server programmatically started or embedded
        --------------------- --------------------------------------------
        E E
        -------------------------------------------------------------------

        >= Java 7

        Server from cmd line Server programmatically started or embedded
        --------------------- --------------------------------------------
         

        No property| R | E
        specified | |

         

        true | E | E

         

        false | R | R

         
        -----------------------------------------------------------------
        • Admin guide topic: "Controlling database file access"

        Sentence two:

        "This means that by default, other operating system accounts will have
        no access to directories or files created by Derby. This behavior
        enhances security for server-managed databases."

        is perhaps not specific enough: "by default" here refers to the case
        where the server is started from the command line only.

        Maybe we should include a matrix here to?

        Show
        Dag H. Wanvik added a comment - Patch comment: Refman topic > derby.storage.useDefaultFilePermissions > Function > > If you run with Java SE 7, and if you start the Derby Network Server > from the command line, access to databases and to other Derby files is > by default restricted to the operating system account that started the > Network Server. File access is not restricted for embedded databases > or for databases managed by servers that are started programmatically > inside application code using the Derby API. "If you run with Java SE 7" -> "If you run with Java SE 7 or newer" I'd say "database files and other files created by Derby", i.e. derby.properties is a Derby file but not created by Derby and its access will not be modified. As for the logic here, I wonder if we should describe this in another way to make it less confusing, maybe a matrix? (sorry it doesn't look good in here but you get the idea File access determined by: E: access controlled entirely by OS environment of JVM, e.g. effective Unix umask or Window default permissions R: Derby restricts access to the operating system account that started the JVM <= Java 6 Server from cmd line Server programmatically started or embedded --------------------- -------------------------------------------- E E ------------------------------------------------------------------- >= Java 7 Server from cmd line Server programmatically started or embedded --------------------- --------------------------------------------   No property| R | E specified | |   true | E | E   false | R | R   ----------------------------------------------------------------- Admin guide topic: "Controlling database file access" Sentence two: "This means that by default, other operating system accounts will have no access to directories or files created by Derby. This behavior enhances security for server-managed databases." is perhaps not specific enough: "by default" here refers to the case where the server is started from the command line only. Maybe we should include a matrix here to?
        Hide
        Dag H. Wanvik added a comment - - edited

        Maybe something like this can help?

        "Controlling database file access

        When Java creates new files, the visibility (access) of the new file
        is normally determined by the JVM's environment and the file's location only,
        cf. umask on Unix/Linux and default file permissions on Windows NTFS.

        On Java 7 and newer, Derby may (further) restrict the file permissions
        to the OS account that started the Java process, which is the minimum
        needed for operation. This means that other operating system accounts
        will have no access to directories or files created by Derby. This can
        be helpful in enhancing default security for database files.
        "

        The exact behavior is determined by two factors: how the Derby engine
        is started, and the the presence of (or not) and given value of the Java property
        derby.storage.useDefaultFilePermissions.

        The following matrix shows which approach is used:

        <matrix>

        For more information, see "derby.storage.useDefaultFilePermissions" in
        the Derby Reference Manual.

        Show
        Dag H. Wanvik added a comment - - edited Maybe something like this can help? "Controlling database file access When Java creates new files, the visibility (access) of the new file is normally determined by the JVM's environment and the file's location only, cf. umask on Unix/Linux and default file permissions on Windows NTFS. On Java 7 and newer, Derby may (further) restrict the file permissions to the OS account that started the Java process, which is the minimum needed for operation. This means that other operating system accounts will have no access to directories or files created by Derby. This can be helpful in enhancing default security for database files. " The exact behavior is determined by two factors: how the Derby engine is started, and the the presence of (or not) and given value of the Java property derby.storage.useDefaultFilePermissions. The following matrix shows which approach is used: <matrix> For more information, see "derby.storage.useDefaultFilePermissions" in the Derby Reference Manual.
        Hide
        Kim Haase added a comment -

        Thanks, Dag, I will work on this. As for whether the property is static or dynamic, you say that it could be dynamic, but "Changing the value dynamically is not recommended." It appears from the table in http://db.apache.org/derby/docs/dev/ref/crefproper22250.html that if a property is system-wide only, it is always static. So I think it would make sense to document it that way, if it's not a good idea to change it dynamically anyway.

        Show
        Kim Haase added a comment - Thanks, Dag, I will work on this. As for whether the property is static or dynamic, you say that it could be dynamic, but "Changing the value dynamically is not recommended." It appears from the table in http://db.apache.org/derby/docs/dev/ref/crefproper22250.html that if a property is system-wide only, it is always static. So I think it would make sense to document it that way, if it's not a good idea to change it dynamically anyway.
        Hide
        Kim Haase added a comment -

        Attaching a second patch, DERBY-5442-2.diff and DERBY-5442-2.zip, with changes to most of the files in the previous patch, if only to change "7" to "7 or later". I hope I made appropriate changes in the Admin Guide and Reference Manual topics. I changed the language in the Admin Guide along the lines you suggested, Dag, but did not do the same in the reference topic except to add the tables. It's good to have somewhat different language in each, since the Admin Guide topic is more conceptual and the other is just focused on the property.

        Thanks again for the review!

        Show
        Kim Haase added a comment - Attaching a second patch, DERBY-5442 -2.diff and DERBY-5442 -2.zip, with changes to most of the files in the previous patch, if only to change "7" to "7 or later". I hope I made appropriate changes in the Admin Guide and Reference Manual topics. I changed the language in the Admin Guide along the lines you suggested, Dag, but did not do the same in the reference topic except to add the tables. It's good to have somewhat different language in each, since the Admin Guide topic is more conceptual and the other is just focused on the property. Thanks again for the review!
        Kim Haase made changes -
        Attachment DERBY-5442-2.diff [ 12522455 ]
        Attachment DERBY-5442-2.zip [ 12522456 ]
        Hide
        Kim Haase added a comment -

        Since I know this patch can't be reviewed until April 30 at least, I will commit it, since it is in fairly close to final form. Further edits are likely.

        Show
        Kim Haase added a comment - Since I know this patch can't be reviewed until April 30 at least, I will commit it, since it is in fairly close to final form. Further edits are likely.
        Hide
        Kim Haase added a comment -

        Awaiting further comments, committed patch DERBY-5442-2.diff to documentation trunk at revision 1328459.

        Show
        Kim Haase added a comment - Awaiting further comments, committed patch DERBY-5442 -2.diff to documentation trunk at revision 1328459.
        Hide
        Knut Anders Hatlen added a comment -

        As to whether we should document the property as static or dynamic, I think we shouldn't say it's static if it's in fact dynamic. The current version of the docs says "This property is static; if you change it while Derby is running, the change does not take effect until you reboot." If someone reads this and then changes the property while Derby is running, they will be surprised.

        In the derby.storage.useDefaultFilePermissions topic, it might be worth mentioning that the property will only have effect on files created after the property was set. Existing files in the database keep their original permissions.

        Show
        Knut Anders Hatlen added a comment - As to whether we should document the property as static or dynamic, I think we shouldn't say it's static if it's in fact dynamic. The current version of the docs says "This property is static; if you change it while Derby is running, the change does not take effect until you reboot." If someone reads this and then changes the property while Derby is running, they will be surprised. In the derby.storage.useDefaultFilePermissions topic, it might be worth mentioning that the property will only have effect on files created after the property was set. Existing files in the database keep their original permissions.
        Hide
        Dag H. Wanvik added a comment -

        +1 to Knut's comments.

        Show
        Dag H. Wanvik added a comment - +1 to Knut's comments.
        Hide
        Dag H. Wanvik added a comment -

        Thanks for patch 2, Kim! Only one comment, but cf Knut's suggestions above also. I think in ref man section derby.storage.useDefaultFilePermissions, we should remove the texual explanation in the bullets:

        (quote)
        "- f you set the property to true, the Network Server uses whatever default file permissions the user has set for their system.

        • If you set the property to false, access is limited not only for files created by the Network Server when it is started from the command line, but also for embedded databases and for databases managed by servers which are started programmatically inside application code using the Derby API. This enhances security for all database files.
          "

        since this is better explained below in the matrix. [Btw: The bullets points speak of "the network server", which is misleading here, since by setting the property, Derby will use specified behavior also in embedded mode (as shown in the matrix)].

        Show
        Dag H. Wanvik added a comment - Thanks for patch 2, Kim! Only one comment, but cf Knut's suggestions above also. I think in ref man section derby.storage.useDefaultFilePermissions, we should remove the texual explanation in the bullets: (quote) "- f you set the property to true, the Network Server uses whatever default file permissions the user has set for their system. If you set the property to false, access is limited not only for files created by the Network Server when it is started from the command line, but also for embedded databases and for databases managed by servers which are started programmatically inside application code using the Derby API. This enhances security for all database files. " since this is better explained below in the matrix. [Btw: The bullets points speak of "the network server", which is misleading here, since by setting the property, Derby will use specified behavior also in embedded mode (as shown in the matrix)] .
        Hide
        Kim Haase added a comment -

        Thanks, Knut and Dag. I was totally uncertain about the scope and static/dynamic values for the property, so I appreciate your taking a close look at those.

        The Network Server references may be because the topic is in the Admin Guide. I should probably clarify there that the behavior applies to embedded mode too.

        Next week I'll file another patch.

        Show
        Kim Haase added a comment - Thanks, Knut and Dag. I was totally uncertain about the scope and static/dynamic values for the property, so I appreciate your taking a close look at those. The Network Server references may be because the topic is in the Admin Guide. I should probably clarify there that the behavior applies to embedded mode too. Next week I'll file another patch.
        Hide
        Kim Haase added a comment -

        I'd better ask this explicitly: Can the derby.storage.useDefaultFilePermissions property be set on a database or only system-wide?

        So far, all properties that are system-wide only are also static. If this property is dynamic, it would normally be settable on a database too.

        Also, if we say that it's dynamic but it's not a good idea to set it dynamically, what is the explanation for why it's not a good idea?

        Show
        Kim Haase added a comment - I'd better ask this explicitly: Can the derby.storage.useDefaultFilePermissions property be set on a database or only system-wide? So far, all properties that are system-wide only are also static. If this property is dynamic, it would normally be settable on a database too. Also, if we say that it's dynamic but it's not a good idea to set it dynamically, what is the explanation for why it's not a good idea?
        Hide
        Knut Anders Hatlen added a comment -

        > Can the derby.storage.useDefaultFilePermissions property be set on a database or only system-wide?

        Only as a system property or as an application property in derby.properties, if I read the code correctly.

        > Also, if we say that it's dynamic but it's not a good idea to set it dynamically, what is the explanation for why it's not a good idea?

        Since the property will only affect the permissions of files created after the property was set, setting it dynamically will make some database files restricted and others not restricted. One will have to shut down the database and run operating system commands (like chmod) to change the permissions of already created files. Although it's not a problem in itself that the permissions are not consistent, users will typically want either all files in the database or no files to be restricted, not a mix of restricted/unrestricted files.

        Show
        Knut Anders Hatlen added a comment - > Can the derby.storage.useDefaultFilePermissions property be set on a database or only system-wide? Only as a system property or as an application property in derby.properties, if I read the code correctly. > Also, if we say that it's dynamic but it's not a good idea to set it dynamically, what is the explanation for why it's not a good idea? Since the property will only affect the permissions of files created after the property was set, setting it dynamically will make some database files restricted and others not restricted. One will have to shut down the database and run operating system commands (like chmod) to change the permissions of already created files. Although it's not a problem in itself that the permissions are not consistent, users will typically want either all files in the database or no files to be restricted, not a mix of restricted/unrestricted files.
        Hide
        Kim Haase added a comment -

        Thanks for all the help. I'm attaching DERBY-5442-3.diff, DERBY-5442-3.stat, and DERBY-5442-3.zip, with additional changes to just two files:

        M src/ref/crefproper22250.dita
        M src/ref/rrefproperdefaultfileperms.dita

        Please let me know if more work is needed on this.

        Show
        Kim Haase added a comment - Thanks for all the help. I'm attaching DERBY-5442 -3.diff, DERBY-5442 -3.stat, and DERBY-5442 -3.zip, with additional changes to just two files: M src/ref/crefproper22250.dita M src/ref/rrefproperdefaultfileperms.dita Please let me know if more work is needed on this.
        Kim Haase made changes -
        Attachment DERBY-5442-3.diff [ 12526014 ]
        Attachment DERBY-5442-3.stat [ 12526015 ]
        Attachment DERBY-5442-3.zip [ 12526016 ]
        Hide
        Dag H. Wanvik added a comment -

        Thanks, Kim! +1

        Show
        Dag H. Wanvik added a comment - Thanks, Kim! +1
        Hide
        Kim Haase added a comment -

        Thanks again, Dag!

        Committed patch DERBY-5442-3.diff to documentation trunk at revision 1336732.

        Show
        Kim Haase added a comment - Thanks again, Dag! Committed patch DERBY-5442 -3.diff to documentation trunk at revision 1336732.
        Kim Haase made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Issue & fix info Patch Available [ 10102 ]
        Resolution Fixed [ 1 ]
        Hide
        Kim Haase added a comment -

        Changes have appeared in Latest Alpha Manuals.

        Show
        Kim Haase added a comment - Changes have appeared in Latest Alpha Manuals.
        Kim Haase made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Gavin made changes -
        Workflow jira [ 12636162 ] Default workflow, editable Closed status [ 12796638 ]

          People

          • Assignee:
            Kim Haase
            Reporter:
            Dag H. Wanvik
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development