Will org.apache.derby.iapi.jdbc be the correct location for EmbedSQLWarning? It's used by the language code, but I understand a goal of yours is to not have the language code depend on the jdbcapi layer.
EmbedSQLWarning is really a factory class to return SQLWarning objects, and it should return plain java.sql.SQLWarning objects rather than a Derby specific implementation. Assume a re-name to SQLWarningFactory, then it would depend on only on the message service. Since this is really related to error handling should SQLWarningFactory belong in org.apache.derby.iapi.error?
I think the EmbedSQLWarning cleanup should be a separate issue.
PublicAPI may also benefit from any improvement not to use EmbedSQLException, thus it may be worth addressing that first.
Also the replication code is using PublicAPI but seems to be changing its approach, thus it may be better to hold off this patch until that code has been updated.