[IGNITE-13154] Introduce the ability for a user to manage binary types - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.9
Component/s: binary
Labels:
None

Release Note:
Adds commands to manage cluster binary metadata to control.sh command line utility.
Ignite Flags:

Docs Required, Release Notes Required

Description

We need a way to change schema (including unsupported changes, such as column type change) without cluster restart. This is for the case when all data associated with the binary type has been removed, so removing the old schema is safe.

Now users must stop the cluster and remove the folder with binary metadata manually.

The proposed way is to introduce internal API to manage binary types and public command line interface (via control.sh). That way one can remove a cache from the cluster, then unregister corresponding binary types, then launch a new version of an application that would register the new schema and reload the data.

The current implementation has restrictions:

all cluster nodes must support remove type feature.
the cluster must not contains data with type to remove.
operation of the update type must not be launched on the cluster for the type to remove (operation examples: put into cache, BinaryObjectBuilder#build).
client nodes process metadata operation asynchronously so type may be removed at the client after any delay after the remove type operation is completed.
if the node that contains the old version of the type joins to the cluster where type was removed the type is propagated to cluster metadata (because metadata tombstone not supported).
if the node that contains the old version of the type cannot join to the cluster where type was removed and then updated to the new version (because metadata versioned tombstone not supported).

So, user scenarios looks like:

Be sure that all server nodes supports remove type feature.
Remove caches contains the data with type to remove.
Stops the client node with older version.
Stops all operation with type to remove (don't create binary objects, don't run compute jobs with type to remove).
Remove the type on the stable topology (and production destination topolog).
Waits any delay (depends on the cluster size and clients count)
Produce operations with new version of the type.

Proposed command line interface
New commands (all commands are experimental, i.e. require IGNITE_ENABLE_EXPERIMENTAL_COMMAND=true env variable to be set):

--meta list prints info about all available binary types:
typeId=<ID>, typeName=<name>, fields=<fields_count>, schemas=<schemas_count>, isEnum=<bool>

--meta details (-- typeId <ID>| --typeName <name>) prints detailed info info about specified type. The type may be specified by type name or type ID.
output example:

typeId=0x1FBFBC0C (532659212)
typeName=TypeName1
Fields:
  name=fld3, type=long[], fieldId=0x2FFF95 (3145621)
  name=fld2, type=double, fieldId=0x2FFF94 (3145620)
  name=fld1, type=Date, fieldId=0x2FFF93 (3145619)
  name=fld0, type=int, fieldId=0x2FFF92 (3145618)
Schemas:
  schemaId=0x6C5CC179 (1818018169), fields=[fld0]
  schemaId=0x70E46431 (1894016049), fields=[fld0, fld1, fld2, fld3]

--meta remove (-- typeId <ID>| --typeName <name>) [\-\-out <file_name>] removes metadata for specified type form cluster and saves the removed metadata to the specified file. If the file name isn't specified the output file name is: <typeId>.bin
The command requires confirmation.
N.B.: The all session of thin clients (ODBC, JDBC, thin client) are closed (to cleanup local cache of the binary metadata).
--meta update --in <file_name>] update cluster metadata from specified file (file name is required)
The command requires confirmation.