Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4835

IllegalArgumentException: Wrong FS: s3, expected: hdfs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: v3.1.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      I have encountered a problem when I use cube panner to optimize cube and build job,  I set kylin.env.hdfs-working-dir to S3, Xiaoxiang Yu fixed the issue in other version on KYLIN-4298,
      What should I do repair this problem in v3.1.0? thanks
      Detail

      Error: java.lang.IllegalArgumentException: Wrong FS: s3://teye-kylin-storage/kylin/kylin_metadata/kylin-e6d2bf54-f030-f433-70e4-620316120c6d/ods_mobile_event_cube/optimize/cuboid/base_cuboid, expected: hdfs://ip-10-90-9-34.eu-west-1.compute.internal:8020
      	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:669)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:214)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1440)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1437)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1452)
      	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1440)
      	at org.apache.kylin.engine.mr.steps.ConvergeCuboidDataReducer.doCleanup(ConvergeCuboidDataReducer.java:88)
      	at org.apache.kylin.engine.mr.KylinReducer.cleanup(KylinReducer.java:103)
      	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:179)
      	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:635)
      	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
      	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
      	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
      	at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:223)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
      	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
      	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:178)
      	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

       
       
       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Anguia Anguia
              Reviewers:
              Xiaoxiang Yu
              Testers:
              Anguia
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: