Details
-
Improvement
-
Status: Reopened
-
Critical
-
Resolution: Unresolved
-
4.2.0
-
None
-
Security Level: Public (Anyone can view this level - this is the default.)
-
None
-
branch: javelin (or master if merge happens)
Description
A new tool/class called DatabaseCreator is introduced in javelin, the idea is that DatabaseCreator can be used by mgmt server, cloud-setup-database script, maven (developer/pom.xml) and possibly by plugins in future to:
1. Initialize database using db.properties file
2. Run sql scripts
3. Run database upgrades
Presently, this has been fixed in javelin, in cloud-server and is used by maven to deploydb, the task is to fix that same for cloud-setup-databases.
Current help doc, usage and options:
DatabaseCreator creates the database schema by removing the
previous schema, creating the schema, and running
through the database updaters.
Usage: DatabaseCreator [options] [db.properties file] [schema.sql files] [database upgrade class]
Options:
--database=a,b comma separate databases to initialize, use the db name in db.properties defined as db.xyz.host, xyz should be passed
--rootpassword=password, by default it will try with an empty password
--dry or -d, this would not run any process, just does a dry run
--verbose or -v to print running sql commands, by default it won't print them
--help or -h for help
The issue now is to make cloudstack-setup-databases use DatabaseCreator and remove db upgrading logic from mgmt server and move it to DatabaseCreator.
Let's talk about the present database deployment and the issues:
- We've two different routes to deploydb, for developers it's
DatabaseCreator now (used to be a maven plugin), for sysadmins it's
cloudstack-setup-database - Upgrades are done by mgmt server, this has issues when there is a
clustered mgmt server, i.e. multiple mgmt servers sharing the same
database, we do upgrade by:
a. Turn down one mgmt server, upgrade it, start it. This mgmt
server checks in db the cloudstack version, upgrades the db
b. Sysadmins meanwhile upgrades all the other mgmt server
c. When all mgmt servers are upgraded, they ask the cluster
manager to run the cleanup. The whole process is automated, an error
can introduce whole sets of issues.
How will DatabaseCreator help:
- Would give a single tool used by both developer and sysadmins. Agony
of the sysadmins would be much better understood by developers.
Sysadmins would still use cloudstack-setup-database which would be
wrapper around DatabaseCreator - Upgrade would be driven by sysadmin, giving them explicit power to
backup database, deploy etc. For example in case of multiple
(clustered) mgmt server, the upgrade would be like:
a. Upgrade database using DatabaseCreator, this step do a backup
and would just upgrade the db such that it's backward compatible to
the old database and old mgmt servers can still run on it. When this
succeeds, system can proceed. Transaction update would provide a way
to rollback.
b. Sysadmins would stop, upgrade and start mgmt server one by one.
c. Sysadmins would run the final cleanup operation, this would do a
final round of cleaning up. If everything goes well, they just did a
rolling CloudStack upgrade with little to no downtime (no downtime as
at least one mgmt server was running
TODOs for 4.1:
- We stick to major db version as 4.0, create-schema is not touched
unless it's a major version. So, next change is to be done only for
5.0. - We move out any 4.1 changes to 4041 upgrade sql file and have a
upgrade path (a upgrading logic/class). - Fix cloudstack-setup-databases, fix packaging.
For 4.2/future:
- Refactor out code that does upgrading in mgmt server to a separate
artifact which would also have DatabaseCreator. - Implement tool so it supports the operations as discussed above
- Replace the current pythonic implementation with DatabaseCreator,
have same routes for both developers and sysadmins
NOTE TO DEVELOPERS:
- WE NEVER CHANGE create-schema.sql
- WE TEST AND IMPLEMENT UPGRADE PATHS from version a to version b
- WE CLEANUP PROPERLY via our upgrade sql files and upgrade classes