I've experimented with some code changes, see the attached patch
experiment.diff, to see if it would work to implement this along the
lines suggested above. Here's a description of how the patch works and
the behaviour we will get if we go for that approach:
A new property called derby.authentication.builtin.algorithm is
introduced. This is a dynamic property that can be set at database or
system level. The property names a digest/hash algorithm that must be
supported by a security provider registered in the JVM.
If this property is set to a non-empty string, calls to
SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY that change a password (that
is, set a derby.user.* property) will use the algorithm specified by
the property to hash the password before storing it in the
database. Otherwise, the old algorithm (based on SHA-1) is used.
Since the hashing happens when a derby.user.* property is set, a
change in which algorithm to use will only take effect on passwords
that are changed after the algorithm was changed. This means that a
database may contain passwords that are hashed using different
algorithms. To update all the stored passwords to use the new
algorithm, one has to call SYSCS_SET_DATABASE_PROPERTY again on all
derby.user.* properties after the
derby.authentication.builtin.algorithm property has been set.
If the derby.authentication.builtin.algorithm specifies an algorithm
that is not known by any of the registered security providers, an
SQLException with SQLState XJ001 (java exception) will be raised if a
password is attempted modified later. The SQLException is chained to a
If a stored password token for a user has been hashed with an
algorithm that is not available on the current platform (could for
example happen if the database was created on another platform with a
different set of security providers), an SQLException with SQLState
XJ001 (java exception) will be raised when validating the credentials
for that user. The SQLException is chained to a
We may want more specific messages in these situations. Currently, the
message looks like this if you specify a non-existing algorithm, for
ij> call syscs_util.syscs_set_database_property('derby.user.kah', 'abc');
ERROR XJ001: Java exception: 'java.security.NoSuchAlgorithmException: DOES-NOT-EXIST MessageDigest not available'.
If derby.authentication.builtin.algorithm is not set or an empty
string, there should be no change in behaviour.
Otherwise, you will get the new behaviour. The new behaviour should be
mostly transparent to the users. However, you will no longer be able
to connect from the client driver with securityMechanism=8 (strong
password substitution). This is because the strong password
substitution security mechanism needs to use the same algorithm itself
to produce a password token that can be compared against the token
stored in the database.
Adding support for this on the client is not straightforward, since
the client would need to be told by the server which algorithm to use,
and this would probably require a protocol extension. It is further
complicated by the fact that the passwords can be stored with
different hash algorithms for different users. Anyways, that's for
another issue to address, if someone wants that functionality.
Since earlier versions of Derby don't understand the new format of the
stored password tokens, derby.authentication.builtin.algorithm is
ignored if the data dictionary is not at version 10.6 or higher. The
old algorithm will be used instead in that case. This allows you to
change the password of a user while running in soft upgrade mode and
still be able to log in as that user if you move back to an older
version of Derby.
If the database has been upgraded (with a full upgrade) so that the
data dictionary is at version 10.6 or later, or if the database was
created with Derby version 10.6 or later, setting the
derby.authentication.builtin.algorithm will modify the behaviour as
described in the sections above.
The code changes are limited to two classes (with the exception of the
definition of a constant in a third one).
The method map(), which does the mapping from plaintext password to
hashed password, is extended with logic to check the
derby.authentication.builtin.algorithm property and the data
dictionary version in order to choose which algorithm to use.
The method authenticateUser() is extended with logic so that it
understands which algorithm has been used to hash a stored
password. It applies the same algorithm on the user-supplied password
and compares it with the one stored in the database.
The existing code in these two classes already had a mechanism to
distinguish between different hashing approaches. It did this by
prefixing the stored hash with an identifier telling which approach
was used. The code in the patch piggybacked on that mechanism and
prefixed the tokens with a different identifier. Also, the name of the
algorithm (for instance SHA-256 or MD5) is appended to the token, with
a colon separating the hash and the name of the algorithm.
One somewhat confusing detail is that the old authentication scheme is
called the new authentication scheme, both in comments and names of
constants. We may want to rename it to something that's more neutral
to where it is on the evolutionary path. SHA-1 authentication scheme,
For the new authentication scheme implemented here, I tentatively
picked the name "configurable hash authentication scheme".
- Internal differences between the schemes *
There are some differences in how the hashes/digests are generated in
the two schemes, apart from the obvious that the old scheme always
uses SHA-1 and the new scheme can use a variety of algorithms.
- Translating passwords into byte arrays
The old scheme uses StringUtil.toHexByte() to convert the password to
a byte array. This code however assumes that all characters are within
the ISO-8859-1 range by only looking at the eight lower bits. It
doesn't break when characters outside this range is used, but it does
not give you any benefit from the extra bits/bytes you've provided
either. Therefore, I used String.getBytes("UTF-8") instead, which is
simpler and uses all bits regardless of which Unicode characters you
have in your password.
The old scheme created a hash from the password alone. This means that
if two users have the same password, they will have the same hashed
password. In the new scheme, the user name and the password are
concatenated before applying the hash function. This means that two
users with the same password will (most likely) not have the same
hashed password, which makes a dictionary attack more time
consuming. Although it's not strictly necessary within the context of
this issue, it sounded like a cheap way to enhance the security while
we're first at it.
For those interested, I found this blog entry well-written and an easy
to understand introduction to the technique of salting password
(The current scheme in Derby is similar to scenario 1, described in
that entry, and the proposed new scheme is similar to scenario 2.)
Comments and suggestions to improve this proposal are welcome!