Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4406

Wrong FS Exception when warehouse and scratchdir are on different FS

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.9.3, 0.10.2
    • None
    • None

    Description

       

      tez.TezTask: Failed to execute tez graph.
      java.lang.IllegalArgumentException: Wrong FS: hdfs://mycluster/tmp/hive/hive/<redacted>/tez-dag.pb1, expected: wasb://<redacted>@<redacted>.blob.core.windows.net
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:743) ~[hadoop-common-3.1.1.jar:?]
              at org.apache.hadoop.fs.azure.NativeAzureFileSystem.checkPath(NativeAzureFileSystem.java:1393) ~[hadoop-azure-3.1.1.jar:?]
              at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:893) ~[hadoop-common-3.1.1.jar:?]
              at org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:676) ~[tez-api-0.9.1.4.1.8.29.jar:0.9.1.4.1.8.29]
              at org.apache.tez.client.TezClient.submitDAG(TezClient.java:588) ~[tez-api-0.9.1.4.1.8.29.jar:0.9.1.4.1.8.29]
              at org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:557) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:220) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2707) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2378) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2051) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1751) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1745) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157) ~[hive-exec-3.1.0.jar:3.1.0]
              at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) ~[hive-service-3.1.0.jar:3.1.0]
              at org.apache.hive.service.cli.operation.SQLOperation.access$600(SQLOperation.java:87) ~[hive-service-3.1.0.jar:3.1.0]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:323) ~[hive-service-3.1.0.jar:3.1.0]
              at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_312]
              at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_312]
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732) ~[hadoop-common-3.1.1.jar:?]
              at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:341) ~[hive-service-3.1.0.jar:3.1.0]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_312]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_312]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_312]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_312]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_312]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_312]
              at java.lang.Thread.run(Thread.java:748) [?:1.8.0_312]

       

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            gupta.nikhil0007 Nikhil Gupta
            gupta.nikhil0007 Nikhil Gupta
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h

                Slack

                  Issue deployment