Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4485

Create a self service interface for cube migration

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • v3.1.0
    • Tools, Build and Test
    • None
    • Sprint 52

    Description

      Current cube migration process is as follows:

       

      There're a few drawbacks:

      • kylin admin has to manually check all related aspects to decide whether a cube is good for migration. If cube design is not good, there'll be many interaction between users & kylin admins
      • kylin admin has to login to the backend server and then run a command to do the migration, which is tricky and easy to make mistakes
      • there's little compatibility check between source metadata & destination cluster.

      A self service interface will refine the migration process as follows:

      We can see with the new migration process, there'll be few interactions between users & kylin admins and much less work for kylin admins.

       

      For this new feature, add some new properties:

      • kylin.cube.migration.enabled
        Specifies whether to show 'Migrate', 'Approve Migration' and 'Reject Migration' in "Action" drop-down list for cube management, the default value is false;
      • kylin.metadata.model-schema-updater-checker-enabled
        Specifies whether to check for model schema update when check check model compatibility, the default value is false;
      • kylin.metadata.able-change-string-to-datetime
        Specifies whether to enable to convert String to Date or Time when check table schema update, the default value is false;
      • kylin.cube.rowkey-encoding-auto-convert
        Specifies whether to enable to convert date/time dictionary on date/time/varchar column to DimensionEncoding implicitly, the default value is true;
      • kylin.server.self-discovery-enabled
        Specifies whether to enable to discover kylin instance from zookeeper, the default value is false;
      • kylin.cube.migration.expansion-rate
        Specifies the threshold value of expansion rate when check the quality of the cube, the default value is 5;
      • kylin.cube.migration.query-generator-max-dimension-number
        Specifies the max number of dimension column when generate sql for checking the quality of the cube, the default value is 3;
      • kylin.cube.migration.rule-query-latency-enabled
        Specifies whether to check query latency when check the quality of the cube, the default value is true;
      • kylin.cube.migration.query-latency-seconds
        Specifies the threshold value of query latency in seconds when check the quality of the cube, the default value is 2;
      • kylin.cube.migration.query-latency-max-threads
        Specifies the max number of query threads to execute sql when check the quality of the cube, the default value is 5;
      • kylin.cube.migration.query-latency-iteration
        Specifies the max number of query sql to be generated when check the quality of the cube, the default value is 5;
      • kylin.cube.migration.source-address
        Specifies the source address of kylin instance;
      • kylin.cube.migration.target-address
        Specifies the target address of kylin instance;
      • kylin.cube.notification-mail-suffix
        Specifies the default suffix address of email for sending notifications.

      Attachments

        1. cube-migration-new.png
          27 kB
          Zhong Yanghong
        2. cube-migration-old.png
          28 kB
          Zhong Yanghong

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              yaho Zhong Yanghong
              yaho Zhong Yanghong
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: