Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-13154

Introduce the ability for a user to manage binary types

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9
    • Component/s: binary
    • Labels:
      None
    • Release Note:
      Adds commands to manage cluster binary metadata to control.sh command line utility.
    • Ignite Flags:
      Docs Required, Release Notes Required

      Description

      We need a way to change schema (including unsupported changes, such as column type change) without cluster restart. This is for the case when all data associated with the binary type has been removed, so removing the old schema is safe.

      Now users must stop the cluster and remove the folder with binary metadata manually.

      The proposed way is to introduce internal API to manage binary types and public command line interface (via control.sh). That way one can remove a cache from the cluster, then unregister corresponding binary types, then launch a new version of an application that would register the new schema and reload the data.

      The current implementation has restrictions:

      • all cluster nodes must support remove type feature.
      • the cluster must not contains data with type to remove.
      • operation of the update type must not be launched on the cluster for the type to remove (operation examples: put into cache, BinaryObjectBuilder#build).
      • client nodes process metadata operation asynchronously so type may be removed at the client after any delay after the remove type operation is completed.
      • if the node that contains the old version of the type joins to the cluster where type was removed the type is propagated to cluster metadata (because metadata tombstone not supported).
      • if the node that contains the old version of the type cannot join to the cluster where type was removed and then updated to the new version (because metadata versioned tombstone not supported).

      So, user scenarios looks like:

      1. Be sure that all server nodes supports remove type feature.
      2. Remove caches contains the data with type to remove.
      3. Stops the client node with older version.
      4. Stops all operation with type to remove (don't create binary objects, don't run compute jobs with type to remove).
      5. Remove the type on the stable topology (and production destination topolog).
      6. Waits any delay (depends on the cluster size and clients count)
      7. Produce operations with new version of the type.

      Proposed command line interface
      New commands (all commands are experimental, i.e. require IGNITE_ENABLE_EXPERIMENTAL_COMMAND=true env variable to be set):

      • --meta list prints info about all available binary types:
        typeId=<ID>, typeName=<name>, fields=<fields_count>, schemas=<schemas_count>, isEnum=<bool>
      • --meta details (-- typeId <ID>| --typeName <name>) prints detailed info info about specified type. The type may be specified by type name or type ID.
        output example:
        typeId=0x1FBFBC0C (532659212)
        typeName=TypeName1
        Fields:
          name=fld3, type=long[], fieldId=0x2FFF95 (3145621)
          name=fld2, type=double, fieldId=0x2FFF94 (3145620)
          name=fld1, type=Date, fieldId=0x2FFF93 (3145619)
          name=fld0, type=int, fieldId=0x2FFF92 (3145618)
        Schemas:
          schemaId=0x6C5CC179 (1818018169), fields=[fld0]
          schemaId=0x70E46431 (1894016049), fields=[fld0, fld1, fld2, fld3]
        
      • --meta remove (-- typeId <ID>| --typeName <name>) [\-\-out <file_name>] removes metadata for specified type form cluster and saves the removed metadata to the specified file. If the file name isn't specified the output file name is: <typeId>.bin
        The command requires confirmation.
        N.B.: The all session of thin clients (ODBC, JDBC, thin client) are closed (to cleanup local cache of the binary metadata).
      • --meta update --in <file_name>] update cluster metadata from specified file (file name is required)
        The command requires confirmation.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tledkov-gridgain Taras Ledkov
                Reporter:
                tledkov-gridgain Taras Ledkov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6h
                  6h