I think the only change I'm suggesting is that when a new collection is created via bootstrapping and the -DnumShards=3 parameter is passed,
that empty placeholders for shard2 and shard3 be created:
This allows for appropriate errors to be thrown when shards that would normally be needed are not "up".
All new nodes being brought up can now omit the -DnumShards parameter... and the overseer assigns as it normally would.
We should shoot for being able to have pretty dumb nodes on startup that are only told to join a cluster and nothing else... the cluster controls everything.
Do you mean that the new node would automatically discover from the placeholders how many cores it needs to start?
No... by default it would still only be one core. Although we could have a parameter that would start more than one core.
What did you have in mind that should be stored under /collections?
There could be a placeholder for shard2, but if the only thing that uses it is leader election, and it's always created on demand, then it shouldn't be necessary.
In earlier versions I had overseer read the target number of slices from the collection node (/collections/collection1 for example) but that was later removed in favor of the system property.
Right - that still sounds correct as it will fit better with custom sharding and with shard splitting (however that will work).
What should be in charge of creating a new collection? Now you can do it for example through CoreAdminHandler by simply adding a core into a collection that does not yet exist
We should eventually have some API to create a new collection without necessitating the creation of new nodes. Perhaps it's part of the core admin handler? Perhaps Mark has thoughts on this.