Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4213

The new build engine with Spark-SQL

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Future
    • Fix Version/s: None
    • Component/s: Job Engine
    • Labels:
      None

      Description

      1. Use Spark-SQL to compute cuboid, build cuboid A, B, C , Sum(D) is sql "select A B C Sum(D) from table group by A, B, C".
      2. To void many memory errors or other exceptions, we can auto set spark conf with build job.E.g use adaptive execution.
      3. The snapshot table will save a table with parquet format.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hn5092 yiming.xu
                Reporter:
                hn5092 yiming.xu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: