Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3758

Flink Cube Build Engine

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • v3.1.0
    • Flink Engine
    • None

    Description

      Kylin currently supports Apache Spark as the engine for building Cubes. As a next-generation computing engine, Apache Flink is dedicated to the convergence of streams and batches, with superior performance and elegant architecture. Flink and Kylin have many overlapping users in China. If Kylin can support Flink as the engine for building Cube, then this will give many users more choices.

      Attachments

        1.
        		 Create Flink engine module and initialize module structure Sub-task Closed vinoyang
        2.
        		 Flink cubing step : fact distinct Sub-task Closed Harvey Yue
        3.
        		 Flink cubing step : build by layer Sub-task Closed vinoyang
        4.
        		 Flink cubing step : convert to HFile Sub-task Closed Harvey Yue
        5.
        		 Flink cubing step : merge by layer Sub-task Closed vinoyang
        6.
        		 Flink cubing step : merge dictionary Sub-task Closed vinoyang
        7.
        		 Provide Flink binary package download script Sub-task Closed vinoyang
        8.
        		 Support Flink cube engine for Kylin web UI Sub-task Closed vinoyang
        9.
        		 Provide Chinese documents for Flink cubing engine Sub-task Open Yaqian Zhang
        10.
        		 Provide English documents for Flink cubing engine Sub-task Open Yaqian Zhang
        11.
        		 Refactor functionally similar code in Flink and Spark modules Sub-task Open vinoyang
        12.
        		 Implement Flink batch cubing job builder Sub-task Closed vinoyang
        13.
        		 Implement Flink batch merge job builder Sub-task Closed vinoyang
        14.
        		 Fix flink config option wrong key mapping Sub-task Closed vinoyang
        15.
        		 Add flink specific config items for kylin properties configuration files Sub-task Closed vinoyang
        16.
        		 Implement FlinkEntry Sub-task Closed vinoyang
        17.
        		 Implement IFlinkOutput based on HBase Sub-task Closed vinoyang
        18.
        		 Implement IFlinkInput based on Hive Sub-task Closed vinoyang
        19.
        		 Fix dependence issue when debug in local Sub-task Closed Chao Long
        20.
        		 Support more dependency jars in FlinkExecutable for FlinkCubeHFile Sub-task Closed Harvey Yue
        21.
        		 Fix missing dependency when submit flink application to YARN Sub-task Closed vinoyang
        22.
        		 Download required dependencies jar from maven central repository for FlinkCubeHFile Sub-task Closed vinoyang
        23.
        		 Replace dead HADOOP_CLASSPATH in FlinkExecutable Sub-task Closed vinoyang
        24.
        		 Add unit test for FlinkOnYarnConfigMapping Sub-task Closed vinoyang
        25.
        		 Bump Flink version from 1.7.1 to 1.9.0 Sub-task Closed vinoyang
        26.
        		 Support flink engine type in the whole project Sub-task Open vinoyang
        27.
        		 Provide a custom serializer for PercentileCounter Sub-task Closed vinoyang
        28.
        		 Provide more config options for Flink cube engine Sub-task Closed vinoyang
        29.
        		 Fix a CLI option parse error for FlinkMergingDictionary and some little code refactor Sub-task Closed vinoyang
        30.
        		 Fail to merge segments by Flink engine Sub-task Closed nichunen
        31.
        		 Use reduceGroup operator to optimize build cube step Sub-task Closed vinoyang
        32.
        		 Use mapPartition operator to optimize build cube step Sub-task Closed vinoyang
        33.
        		 Provider config options to specify yarn queue and nodelabel Sub-task Closed vinoyang
        34.
        		 Change Flink jar dependence from compile to provided Sub-task Closed Harvey Yue
        35.
        		 Bump Flink version from 1.9.0 to 1.9.2 Sub-task Closed Harvey Yue
        36.
        		 Bump Flink version from 1.9.2 to 1.10.0 Sub-task Reopened Harvey Yue
        37.
        		 Missing data size in web UI with flink cubing Sub-task Closed Harvey Yue
        38.
        		 Add a switch to enable/disable flink cubing step: convert to HFile Sub-task Closed Harvey Yue

        Activity

          People

            yanghua vinoyang
            yanghua vinoyang
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: