Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3461

"metastore.sh refresh-cube-signature" not updating cube signature as expected

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • v2.3.2
    • v2.4.1, v2.5.0
    • Client - CLI
    • None

    Description

      Recently I upgraded kylin from 2.3.1 to 2.3.2, and when I tried to build a cube, the Web UI told me that my cube signature is inconsistent, and suggested that I use the metastore.sh script to refresh all cube signatures. 
      I did what I was told to, but as I can read from the log, the script didn't update any cube signature, and I still received the same error message when I retried to build the cube.
      I did some digging on the source code, and add some logging statements, re-packaged kylin and deployed the more-logging version. I found that when kylin refreshes cube signatures and build new cube segments, it always check whether the cube signature equals to the calculated signature which is calculated by method org.apache.kylin.cube.model.CubeDesc#calculateSignature().

      And as I observed from the log, this method return DIFFERENT RESULT under the above two occasions!

      The Difference lies in that when building cube segments invokes calculateSignature(), it will add "encoding_version" into the "rowkey" json string.

      Attachments

        1. build-cube-log.png
          33 kB
          Yuan Weizhao
        2. log1-calculate-signature-when-building.log
          126 kB
          Yuan Weizhao
        3. log-2-calculate-signature-when-refresh-cube-signature.log
          83 kB
          Yuan Weizhao
        4. refresh-cube-signature-log.png
          117 kB
          Yuan Weizhao

        Issue Links

          Activity

            People

              Na Zhai Na Zhai
              weizhaoy Yuan Weizhao
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: