Thanks for this first tranche of changes, Kim. I can see that this is a big effort indeed. On the whole, these changes look great to me.
I now understand why you asked so many questions about the sample programs. I see that the programs originated in the docs examples. Now that I see that the programs are going to be exposed, I think that we should overhaul them. They don't showcase how easy Derby is to use--and how much easier authentication has become with the new NATIVE mechanism. If you like, I would be happy to rewrite these examples to try to showcase how simple Derby security is becoming. We just need to agree on what points the examples should punch up.
I stumbled over the first paragraph. I would be tempted to reword it slightly:
"If you use NATIVE authentication, then you do not need to set the derby.connection.requireAuthentication property. When creating a database with NATIVE authentication, simply specify a username and password--that user becomes the database owner."
I would recommend some changes to the pre-existing wording. In general, I don't think that the term "user authorizations" conveys any more meaning than the simpler term "authorizations". On the contrary, I think it is easy to be confused by the term "user authorizations" and to suppose that it is yet another kind of authorization scheme in addition to "connection authorizations" and "SQL authorizations". There are only 2 kinds of authorizations and I find that the qualifiers "coarse-grained" and "fine-grained" help me tease them apart.
o Paragraph 2 under "User authorizations":
"Connection authorization specifies the access" -> "Connection authorization specifies the coarse-grained access"
"SQL authorization controls the permissions" -> "SQL authorization controls the fine-grained permissions"
o Paragraph 4 under "User authorizations": Coarse-grained connection authorization and fine-grained SQL authorization are independent of one another. You can enable both of them if you want to. NATIVE authentication does not affect this independence. If you turn on NATIVE authentication, then you can also turn on coarse-grained connection authorization. I have verified this. Here is an attempt to reword this paragraph:
"Attention: If you use NATIVE authentication, then fine-grained SQL authorization is automatically enabled, and by default, all users enjoy full coarse-grained access to the database. In this situation, fine-grained SQL authorization can't be turned off. However, you can still adjust coarse-grained access to the database."
o Paragraph 3 under "User authorization properties": The html output is missing the blank line which I think should introduce the new paragraph that begins with the sentence "The following properties affect authorization:"
o Paragraph 4 under "User authorization properties": This paragraph really confuses me. I think that it is supposed to be talking about coarse-grained connection authorizations, but it uses the ambiguous term "user authorizations". I recommend rewording it:
"If you do not specify the coarse-grained connection authorizations for a specific user ID, then that user ID inherits the database's default coarse-grained connection authorization."
o Bullet 3 under "How user authorization properties work together": The two authorization mechanisms do not override one another. They supplement or refine one another. A user's authorizations end up being the intersection of her coarse-grained and fine-grained permissions. I would reword the first sentence of this bullet as follows:
"The coarse-grained access mode specified by the derby.database.defaultConnectionMode property supplements the permissions that re granted by the owner of a database object."
One specific comment, then a general comment about the pre-existing wording.
o Bullet 1: I would drop the sentence about robustness. I don't think we want to imply that LDAP or user-written mechanisms are less safe.
The pre-existing wording seems a bit confused to me. It creates the impression that authentication controls access to the Derby engine and authorization controls access to individual databases. That's not how it works. Here's how I see it:
1) Authentication determines whether you are a legal user. It establishes your identity.
2) Authorization determines what operations can be performed by you, that is, by your Derby identity.
So far, so good. Those are just generic definitions of authentication and authorization which hold true for lots of software, not just Derby. Now for the tricky bit, which is specific to Derby:
Derby understands two kinds of identity:
A) System-wide identity. Currently, any legal system-wide identity enjoys authorization to perform the following operations:
i) Create databases.
ii) Restore databases.
iii) Shutdown the Derby engine.
B) Database-specific identity. If you are a legal identity in a specific-database, then you may enjoy the following rights:
i) You can connect to that database--provided that coarse-grained connection authorization has not been set to noAccess.
ii) You can shutdown that database, encrypt it, and upgrade it--provided that you are the database owner.
iii) You can create your own SQL objects and write data to your own tables--provided that your coarse-grained connection authorization has not be set to readOnlyAccess.
iv) You can access other SQL objects--provided that the owners have granted you fine-grained SQL access to those objects and provided you have not been limited by coarse-grained readOnlyAccess.
o Bullet 2: I was confused by multiple references to authorization in this sentence. I would reword it to something like this:
"These examples show how to use fine-grained SQL authorization in conjunction with Derby's NATIVE user authentication."
o Paragraph 1 under "Using NATIVE authentication": Again, I would drop the claim that NATIVE authentication is safer than LDAP. Maybe something like this:
"Derby's simplest authentication mechanism is NATIVE authentication."
o Last paragraph under "Using NATIVE authentication": I would clarify when passwords are hashed. I would reword the first sentence of this paragraph:
"Use the derby.authentication.builtin.algorithm property to change how passwords are encrypted when they are stored in SYS.SYSUSERS."
o Last bullet under "Managing users and passwords": I would punch up the purpose of this procedure:
"To change a password" -> "To change her own password"
o Paragraph 3 under "User authentication and SQL authorization client example": I was confused by the last sentence of this paragraph. I couldn't figure out what "format" referred to.
See my introductory comment about these examples: I see that these programs are pre-existing code, which you have reworked to use NATIVE authentication. I think it would be great if the examples could be simplified to highlight how easy it is to set up NATIVE authentication. In particular, because BUILTIN isn't being used anymore, there is no need to call syscs_set_database_property() at all. If the following property is set on the server...
...then the following work will happen automagically:
1) derby.authentication.provider=NATIVE::LOCAL will be set in the database.
2) The SYSADM credentials will be loaded automatically.
3) And there is no need to set databaseOnlyProperties. The database has been secured by NATIVE::LOCAL authentication so there is no way to subvert authentication now.
I also think that we should remove the code which sets coarse-grained connection authorizations. I believe that this material should have been removed when we introduced fine-grained SQL authorizations in 10.2.
Please be patient as we improve these examples. I think that this may take a couple iterations. As I said earlier, if you like, I can create a simpler example from scratch once we agree on what the example should show.
I'm tempted to get rid of this page and just roll its material into the previous examples. Can you think of a reason why we need this extra page?
o Item 4: Again, I think that we only need to recommend SQL authorization. I would reword the first sentence as follows:
"To prevent unauthorized users from accessing databases once they are booted, turn on user authentication and SQL authorization for the database."
o Item 5: I think that this page is about how to configure security in a production environment. I don't think that we should mention BUILTIN authentication at all since it is not secure. I would just reduce this item to the following. If we remove the bit about BUILTIN authentcation, then there's nothing left to this item except the advice to turn on SSL/TLS. But that doesn't make sense for an embedded scenario. So I would reduce this item to the following and move it to the next page (which deals with configuring network security):
"It is also strongly recommended that production systems protect network connections with SSL/TLS."
o Item 3: Again, let's not mention BUILTIN authentication. Strike the last two sentences of this item.
o Item 4: This text assumes that you are using coarse-grained connection authorization rather than fine-grained SQL authorization. Again, I think that we should stop recommending the old-style, Derby-specific coarse-grained mechanism. Instead, we should recommend that people use portable SQL Standard features wherever possible. I would simplify this item as follows:
"Configure SQL authorization for your databases."