Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3802

Kylin build process fails at step "Materialize Hive View in Lookup Tables"

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Resolved
    • Affects Version/s: v2.4.1
    • Fix Version/s: None
    • Component/s: Job Engine
    • Labels:
      None

      Description

      When building several cubes in parallel that use a hive view as source, sometimes they fail at step two (Materialize Hive View in Lookup Tables).

      This is due to since we are using a view as source instead of a table, Kylin intermediate table is created without UUID and so second build drops previous built intermediate table. 

      For example:

      0: jdbc:hive2://bi-horton-hive.internalserver> DROP TABLE IF EXISTS kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW; No rows affected (0.586 seconds)
      
      0: jdbc:hive2://bi-horton-hive.internalserver> CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW LIKE DW.DI_OPERADORES_SMS_VIEW LOCATION 'hdfs://XXXXXX/kylin/kylin_me tadata/kylin-5df95c88-a123-44e0-9b1a-c35ecf1599fb/kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW'; No rows affected (0.308 seconds) 
      
      0: jdbc:hive2://bi-horton-hive.internalserver> ALTER TABLE kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW SET TBLPROPERTIES('auto.purge'='true'); Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Table not found kylin_intermediate_DW_DI_OPERADORES_SMS_VIEW (state=08S01,code=1)
      

       

      If no views are used, Kylin appends an uuid to avoid conflicts.

      DROP TABLE IF EXISTS kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4;
      CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_agencia_cubo_v4_4b6d70dd_e0e4_4247_8949_0adef5c0d6c4

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                inigoml Iñigo Martinez
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: