Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4283

FileNotFound error in "Garbage Collection" step should not break cube building.

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v3.0.0, v2.6.5
    • Component/s: Job Engine
    • Labels:
      None

      Description

       

      Job failed because of FileNotFound. 

      I think Garbage Collection failed should not make the build job failed

      // code placeholder
      java.io.FileNotFoundException: File /user/s_kylin/kylin_zjyprc_bigdata_staging/kylin_zjyprc_bigdata_staging-kylin_metadata/kylin-ff3f6fc3-2211-3593-1ba4-9b5326a05258/cube does not exist.
      	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:773)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:831)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:827)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:838)
      	at org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.dropHdfsPathOnCluster(HDFSPathGarbageCollectionStep.java:95)
      	at org.apache.kylin.storage.hbase.steps.HDFSPathGarbageCollectionStep.doWork(HDFSPathGarbageCollectionStep.java:65)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180)
      	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:180)
      	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

       

        Attachments

          Activity

            People

            • Assignee:
              zhoukangcn ZhouKang
              Reporter:
              zhoukangcn ZhouKang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment