Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3463

Improve optimize job by avoiding creating empty output files on HDFS

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • v2.4.1, v2.5.0
    • None
    • None

    Description

      For steps in an optimize job, like FilterRecommendCuboidDataJob & UpdateOldCuboidShardJob, MultipleOutputs is used to output data into different directories. Therefore, the default output file is empty. LazyOutputFormat should be used to prevent to create zero-sized default output.

      Attachments

        1. APACHE-KYLIN-3463.patch
          3 kB
          Zhong Yanghong

        Activity

          People

            yaho Zhong Yanghong
            yaho Zhong Yanghong
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: